verikun 0.19.0 → 0.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/verikun/SKILL.md +514 -0
- package/CHANGELOG.md +822 -0
- package/README.md +26 -754
- package/dist/cli.js +23 -3
- package/dist/update-check.js +189 -0
- package/dist/version.js +1 -1
- package/example/README.md +100 -0
- package/example/example-test-devicestate.md +53 -0
- package/example/example-test.md +40 -0
- package/package.json +6 -3
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,822 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here. The format is based on
|
|
4
|
+
[Keep a Changelog](https://keepachangelog.com/), and the project follows
|
|
5
|
+
[Semantic Versioning](https://semver.org/).
|
|
6
|
+
|
|
7
|
+
## [Unreleased]
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
- **A `Self-healing in CI` docs page, and a plan-cache step in the reference workflow.** The
|
|
11
|
+
question it answers came from a team running verikun in CI: *should it be allowed to heal a
|
|
12
|
+
drifted step, or should that just fail the build?* The answer was spread across five pages,
|
|
13
|
+
none of which framed it as a policy decision, and no page named the two fields you would
|
|
14
|
+
gate on. It now states plainly that healing **cannot be turned off** (`maxRepairs` is a fixed
|
|
15
|
+
3), and why that is safe: only a selector miss or an ambiguity is healable, an `assert`
|
|
16
|
+
failure is structurally unhealable (it *returns* exit `1` rather than throwing, so the engine
|
|
17
|
+
never sees a healable error), and `give_up` is terminal. Every repair leaves a `model-healed:`
|
|
18
|
+
note in the report, a `modelRepairs` count in `index.json`, and a suggested prose fix — so the
|
|
19
|
+
real choice is *heal and tell you* versus fail, not *heal silently* versus fail. For teams
|
|
20
|
+
that want the strict gate anyway, it ships a `jq` step over the manifest rather than a new
|
|
21
|
+
flag. It opens with a diagram of the split, because the common misreading is that the engine
|
|
22
|
+
runs inside `vk server` — it does not; compile, cache, replay, repair and report assembly are
|
|
23
|
+
all on the CI runner, and the device box holds no model key. The diagram's source prompt,
|
|
24
|
+
model and required post-processing are kept beside the asset in
|
|
25
|
+
`docs/src/assets/ci-architecture-overview.prompt.md`, since Starlight silently strips unknown
|
|
26
|
+
frontmatter keys and would have discarded them.
|
|
27
|
+
|
|
28
|
+
It also corrects a claim two pages made without qualification. **The plan cache is
|
|
29
|
+
`./.verikun/plans/`, relative to the working directory, so a disposable CI runner starts cold
|
|
30
|
+
and recompiles every test on every run** — the "a green suite costs roughly \$0" steady state
|
|
31
|
+
was never reachable in CI as documented. `guides/natural-language-tests.md` and
|
|
32
|
+
`internals/plan-ir-and-engine.md` now say so, `guides/troubleshooting.md` lists it as the
|
|
33
|
+
first thing to check when a CI run costs more than expected, and
|
|
34
|
+
`.github/workflows/suite.yml` — which `guides/remote-devices-and-ci.md` presents as a
|
|
35
|
+
reference to copy — gained `actions/cache` restore/save steps so it stops teaching the
|
|
36
|
+
mistake. The `restore`/`save` split is deliberate: the combined action does not save on a
|
|
37
|
+
failed job, and a red run is precisely the one whose fresh compiles are worth keeping.
|
|
38
|
+
|
|
39
|
+
Two facts documented for the first time: `reference/reports-and-test-runs.mdx` now names
|
|
40
|
+
`tests[].modelRepairs` as its own manifest row (it was folded into a prose list, so the field
|
|
41
|
+
a gate needs had no name in the docs), and the page records that a **100%-cached suite still
|
|
42
|
+
needs a model key** — the provider is preflighted unconditionally because a repair must be
|
|
43
|
+
possible at runtime, so a cache hit with no key still exits `3`.
|
|
44
|
+
|
|
45
|
+
No version bump: no CLI behaviour changed.
|
|
46
|
+
|
|
47
|
+
### Fixed
|
|
48
|
+
- **The release gate stopped understanding `npm pack --json`, which blocked the 0.20.0
|
|
49
|
+
publish.** `scripts/check-package-contents.mjs` read the pack result as
|
|
50
|
+
`JSON.parse(raw)[0]`, but **npm 12 returns an object keyed by package name** where npm ≤ 11
|
|
51
|
+
returned an array — so the script died on `Cannot read properties of undefined (reading
|
|
52
|
+
'files')` after every other release check had passed, with the tarball itself perfectly fine.
|
|
53
|
+
It now accepts either shape.
|
|
54
|
+
|
|
55
|
+
Nothing in CI could have caught it: `ci.yml` runs the check under the Node-bundled npm (10.x
|
|
56
|
+
on Node 20, 11.x on Node 22), while `publish.yml` deliberately installs `npm@latest` because
|
|
57
|
+
trusted publishing needs ≥ 11.5.1 — so the release job, and only the release job, gets
|
|
58
|
+
whatever npm major shipped that morning. That is not a bug to pin away: the fix is for the
|
|
59
|
+
script to survive an npm it has never seen, so an unrecognised shape now prints the npm
|
|
60
|
+
version and the raw output and exits 1, instead of throwing a stack trace at whoever is
|
|
61
|
+
mid-release.
|
|
62
|
+
|
|
63
|
+
No version bump: no CLI behaviour changed.
|
|
64
|
+
|
|
65
|
+
## [0.20.0] - 2026-08-12
|
|
66
|
+
|
|
67
|
+
### Added
|
|
68
|
+
- **`vk doctor` now warns when your verikun is out of date**
|
|
69
|
+
([#59](https://github.com/ddikman/verikun/issues/59)). Two checks alongside the device probes: is
|
|
70
|
+
this CLI behind npm's `latest`, and is the installed Claude Code plugin's skill documentation
|
|
71
|
+
behind this CLI. `SKILL.md` asks the agent to run `vk doctor` once at the start of a session and
|
|
72
|
+
to *tell you* rather than upgrade on your behalf.
|
|
73
|
+
|
|
74
|
+
The skill is the agent-facing *contract*, so a stale copy is worse than no copy: an agent reads it
|
|
75
|
+
and confidently uses a flag the installed CLI no longer has. The reporter's machine had a plugin 8
|
|
76
|
+
minor versions behind its CLI, with nothing anywhere saying so — and that cannot be fixed from the
|
|
77
|
+
plugin side, because marketplace auto-update is off by default for third-party marketplaces, the
|
|
78
|
+
setting lives in the user's own config, and no plugin author can enable it.
|
|
79
|
+
|
|
80
|
+
Both are **warnings**: printed on stderr with the command that fixes them, but they do not change
|
|
81
|
+
doctor's exit code, because being out of date is not the same as a machine that cannot drive a
|
|
82
|
+
device — and `3` is documented as "a machine to fix". (`ToolProbe` gained an `advisory` severity
|
|
83
|
+
for this; a firewalled CI box must not go red either.) The whole check degrades to silence —
|
|
84
|
+
offline, a captive portal's HTML body, an unreadable plugin registry — since an advisory must never
|
|
85
|
+
fail a device run. Opt out with `VERIKUN_NO_UPDATE_CHECK`.
|
|
86
|
+
|
|
87
|
+
- **A `Platform support` page in the docs, and it is now the canonical platform matrix.** Every
|
|
88
|
+
command, state modifier, `device set` key and reporting feature, split **Android** × **iOS**
|
|
89
|
+
and each of those into **physical** and **emulator/simulator**. Previously "does this work on
|
|
90
|
+
my setup?" had no single answer: the facts were spread across nine places in three different
|
|
91
|
+
shapes, no column anywhere covered an Android emulator, and the selector-portability table
|
|
92
|
+
existed verbatim in two files.
|
|
93
|
+
|
|
94
|
+
Three things it states that nothing did before. **A physical iOS device supports none of the
|
|
95
|
+
five device-setting keys** — `simctl` drives simulators only and `idb` covers interaction
|
|
96
|
+
rather than preferences — while `vk device caps --ios` reports the *simulator* answer either
|
|
97
|
+
way, because the capability table is static and only the driver knows what it resolved. So
|
|
98
|
+
`caps` says `dark` is supported and `set` exits `3`. **`stay-awake` is a no-op on a simulator
|
|
99
|
+
but a refusal on a device**, which the previous table flattened into one cell. And the
|
|
100
|
+
**iOS-physical column is asserted from the source rather than measured** — the Flutter fixture
|
|
101
|
+
cannot be installed on one without code signing, so no physical iPhone has been exercised end
|
|
102
|
+
to end; the page says so rather than implying a measurement that never happened.
|
|
103
|
+
|
|
104
|
+
The pages that carried a partial copy now link to it instead: `guides/ios-setup.md`'s
|
|
105
|
+
seven-bullet limitations list and its duplicated selector table, and
|
|
106
|
+
`reference/device-state.md`'s three-column key matrix. `guides/troubleshooting.md` keeps its
|
|
107
|
+
symptom→meaning table, which answers a different question. Also corrects `ios-setup.md`'s
|
|
108
|
+
advice to "prefer `text:`/`desc:`" on iOS, which contradicted the measured `@id`-first rule
|
|
109
|
+
three lines below it in the same file.
|
|
110
|
+
|
|
111
|
+
No version bump: no CLI behaviour changed.
|
|
112
|
+
|
|
113
|
+
- **Docs coverage is now enforced in CI.** `tests/docs-coverage.test.ts` reads `src/cli.ts` and
|
|
114
|
+
`src/device/settings.ts` as text, extracts every dispatchable command (aliases included — they
|
|
115
|
+
are fall-through `case` labels sharing a handler) and every `device set` key, and fails if any
|
|
116
|
+
is missing from `reference/commands.md` or `guides/platform-support.md`.
|
|
117
|
+
|
|
118
|
+
It closes a hole rather than adding ceremony: `pages.yml` is the only job that builds the docs
|
|
119
|
+
site, and therefore the only thing that validates its links, and it is **path-filtered to
|
|
120
|
+
`docs/**`**. A pull request that added a command to `src/cli.ts` and touched no docs file never
|
|
121
|
+
triggered it, so nothing anywhere reported the omission. `ci.yml` runs the unit suite on every
|
|
122
|
+
PR regardless of paths. The extraction asserts a floor on how many commands it found, so a
|
|
123
|
+
regex that silently stops matching fails loudly instead of passing vacuously.
|
|
124
|
+
|
|
125
|
+
`CLAUDE.md`'s versioning section is restated to match: one trigger, three inseparable
|
|
126
|
+
consequences — version bump, changelog entry, **and** the docs page that owns the contract.
|
|
127
|
+
Adding a command is now documented as five edits rather than two.
|
|
128
|
+
|
|
129
|
+
- **The docs home page shows the sidebar again.** It used Starlight's `template: splash`, a
|
|
130
|
+
full-width landing layout that drops the sidebar *and* the table of contents — so the page most
|
|
131
|
+
people land on was the one page with no navigation. Now `template: doc`; the hero is unchanged.
|
|
132
|
+
|
|
133
|
+
- **A documentation site at <https://ddikman.github.io/verikun/>.** An Astro Starlight site in
|
|
134
|
+
`docs/`, deployed to GitHub Pages by the new `.github/workflows/pages.yml` (a pull request
|
|
135
|
+
builds it as a check without deploying). 26 pages across four sections: Getting started
|
|
136
|
+
(installation, a ten-minute first test, driving it from an AI agent), Guides (writing test
|
|
137
|
+
cases, natural-language tests, suites, remote devices & CI, iOS setup, troubleshooting),
|
|
138
|
+
Reference (commands, selectors, auto-wait, global flags, exit codes, environment variables,
|
|
139
|
+
reports & test runs, device state, screenshots, AI plans & models), and Internals
|
|
140
|
+
(architecture, core principles, the plan IR and replay engine, contracts, contributing).
|
|
141
|
+
|
|
142
|
+
Two things existed nowhere before: a **single environment-variable reference** — all 18 were
|
|
143
|
+
scattered across ten README locations — and a **troubleshooting guide** that says what a
|
|
144
|
+
given failure actually means. The `vk server` + GitHub Actions setup is now explained step by
|
|
145
|
+
step rather than linked as a file. On the contributor side, the load-bearing rules that only
|
|
146
|
+
lived beside the code (heal-vs-terminal, run rollover, the plan-cache fingerprint, what is
|
|
147
|
+
and is not recordable, the packaging allowlist) are written down in one place a change can be
|
|
148
|
+
checked against.
|
|
149
|
+
|
|
150
|
+
The site is a **separate npm project** — `docs/package.json` with its own lockfile, and the
|
|
151
|
+
repo root has no `workspaces` key — so root `npm ci` never descends into it and the CLI
|
|
152
|
+
package's zero-runtime-dependency rule is untouched. Nothing under `docs/` is in
|
|
153
|
+
`package.json`'s `"files"`, so the npm tarball is byte-identical.
|
|
154
|
+
`starlight-links-validator` fails the build on a dead internal link, which is what makes the
|
|
155
|
+
README's 24 rewritten anchors safe to move.
|
|
156
|
+
|
|
157
|
+
- **The npm package now ships the agent skill and the `example/*.md` tests.** `npm install -g verikun`
|
|
158
|
+
delivers the CLI *and* `SKILL.md` — the agent-facing contract — in one step, rather than needing a
|
|
159
|
+
separate `npx skills add` / plugin install to obtain it. It lands at
|
|
160
|
+
`.claude/skills/verikun/SKILL.md` inside the package, the same path it has in the repo (shipped,
|
|
161
|
+
not duplicated), which also makes the relative `SKILL.md` links in the published README resolve
|
|
162
|
+
rather than dangle.
|
|
163
|
+
- **Packaging is now verified against the built artifact, not just the config.** New
|
|
164
|
+
`scripts/check-package-contents.mjs` packs the tarball and asserts its real contents — required
|
|
165
|
+
docs present; environment/secret files (`.env*`, `*.pem`/`*.key`/`*.keystore`), the
|
|
166
|
+
contributor-only `create-pr` skill, and `src/`/`tests/` absent. The secrets assertion is a third
|
|
167
|
+
independent layer: the `files` allowlist and `.gitignore`'s `.env*` rules already exclude them, but
|
|
168
|
+
npm's own always-excluded list does *not* cover `.env`, so removing both leaves no warning — and a
|
|
169
|
+
leaked credential can't be unpublished out of mirrors. It runs in CI on
|
|
170
|
+
every PR and as a release gate in `publish.yml` before `npm publish`, where a mistake would
|
|
171
|
+
otherwise burn a version number npm will not let us reuse. This catches what an allowlist check
|
|
172
|
+
structurally cannot: a stray `.npmignore`, a change in how npm treats the `.claude/`
|
|
173
|
+
dot-directory, `.claude/skills` added alongside the exact skill path, or a subdirectory appearing
|
|
174
|
+
under `example/`. That last one is not hypothetical — the allowlist entry is the glob
|
|
175
|
+
`example/*.md` rather than a bare `example` precisely because the directory also holds the Flutter
|
|
176
|
+
e2e fixture app, 74 files of Gradle/Xcode/PNG that no end user installing the CLI should download.
|
|
177
|
+
- `publish.yml` also verifies `package-lock.json` is in sync with `package.json`'s version. `npm ci`
|
|
178
|
+
does not catch this — it errors only on dependency drift, never on the root `version` field.
|
|
179
|
+
|
|
180
|
+
### Fixed
|
|
181
|
+
- **The Claude Code plugin no longer ships the contributor-only `create-pr` skill.** The manifest
|
|
182
|
+
pointed `skills` at the container `.claude/skills/`, and that field only ever *adds* to the scan,
|
|
183
|
+
so every skill in it reached end users — costing roughly 260 always-on tokens per session for one
|
|
184
|
+
they cannot use. It now lists the two user-facing skills explicitly. Naming a single skill would
|
|
185
|
+
not do: that drops `suggest-verikun-improvement`, which the main skill hands off to.
|
|
186
|
+
|
|
187
|
+
- **`CHANGELOG.md` is now included in the published npm package.** `package.json`'s `"files"` is an
|
|
188
|
+
allowlist, and npm force-includes only `package.json`, `README`, `LICENSE`, the `main` file and the
|
|
189
|
+
`bin` file(s) — a changelog is *not* on that list (npm 5/6 included one; modern npm does not).
|
|
190
|
+
Every release up to 0.19.0 therefore published without a changelog, and did so silently: the
|
|
191
|
+
publish succeeded, CI was green, and nothing anywhere reported the omission.
|
|
192
|
+
`tests/package-files.test.ts` now guards the allowlist, including against an entry whose path no
|
|
193
|
+
longer exists.
|
|
194
|
+
|
|
195
|
+
## [0.19.0] - 2026-08-08
|
|
196
|
+
|
|
197
|
+
### Added
|
|
198
|
+
- **`vk device set` — change the device the app runs on, then put it back (#31).** Five settings
|
|
199
|
+
to start: `airplane` (go offline, for retry/error handling), `dark`, `font-scale`, `rotation`
|
|
200
|
+
and `stay-awake`, applied as `key=value` assignments so several land in one call —
|
|
201
|
+
`vk device set dark=on font-scale=1.3 rotation=landscape`. Rounded out by `device get`,
|
|
202
|
+
`device reset` and `device caps`. The design is snapshot → override → restore: `set` records
|
|
203
|
+
what each setting held *before* it changed it, in the run file rather than in memory (every
|
|
204
|
+
`vk` call is its own process, so an in-memory latch could not undo a flow that died), and
|
|
205
|
+
`batch` / `ai` / `suite` restore from a `finally` — so a test that dies between `airplane=on`
|
|
206
|
+
and its reset cannot leave the phone offline. A bare `vk device set` from a shell deliberately
|
|
207
|
+
stays applied until you `vk device reset`.
|
|
208
|
+
- **A `@vk_device` screen on the Flutter fixture, so device settings are *assertable*.** Every
|
|
209
|
+
line on it (`vk_dev_brightness`, `vk_dev_orientation`, `vk_dev_textscale`) is read from
|
|
210
|
+
`MediaQuery` — from the platform, not from app state — so
|
|
211
|
+
`assert @vk_dev_brightness --text dark` proves the app itself observed the change, rather
|
|
212
|
+
than proving only that a value landed in a system database. The fixture also gains a
|
|
213
|
+
`darkTheme`: with only a light theme, `ThemeMode.system` has nothing to switch to and a
|
|
214
|
+
dark-mode check would pass vacuously. `example/example-test-devicestate.md` drives it on
|
|
215
|
+
both platforms, and `tests/e2e/flutter-app.test.ts` covers the platform-divergent half
|
|
216
|
+
(Android-only `rotation`; the iOS exit-3 refusal and its manual equivalent). Font scale is
|
|
217
|
+
asserted as "grew, then was restored" rather than against a literal, because the effective
|
|
218
|
+
ratio for `font-scale=1.3` differs per target — `1.30` on API 31, `1.26` on API 34 (Android
|
|
219
|
+
14 scales large text non-linearly) and `1.35` on iOS (nearest Dynamic Type category). All
|
|
220
|
+
three are captured in `example/flutter-app/README.md`'s measured facts.
|
|
221
|
+
- **A capability table (`src/device/settings.ts`) rather than five ad-hoc commands.** It declares
|
|
222
|
+
each key's value domain and its per-platform support — `supported`, `unsupported`, or `noop` —
|
|
223
|
+
and drives argument validation, the driver switch, `vk device caps` and the `vk ai` plan
|
|
224
|
+
validator from one place. Android supports all five; on an iOS **simulator** `dark` and
|
|
225
|
+
`font-scale` work via `simctl ui` (font scale maps to the nearest Dynamic Type category, and
|
|
226
|
+
the applied category is echoed), `stay-awake` is an honest no-op, and `airplane`/`rotation`
|
|
227
|
+
do not exist at all. Every `unsupported` entry names the manual equivalent and every `noop`
|
|
228
|
+
says why it was unnecessary — both enforced by the unit suite, so a future setting cannot be
|
|
229
|
+
added as a silent dead end.
|
|
230
|
+
- **`vk ai` can drive the device, and a bad setting fails at compile time.** `device` joins
|
|
231
|
+
`KNOWN_COMMANDS` and the grammar, and `validateNode` checks a `device` leaf's keys and values
|
|
232
|
+
during **plan validation** — unlike every other command, whose selector failures are runtime
|
|
233
|
+
facts the engine can heal. A suite asking for `rotation` on iOS therefore fails before the
|
|
234
|
+
first tap rather than twenty steps in, on a device it has already half-modified.
|
|
235
|
+
|
|
236
|
+
### Changed
|
|
237
|
+
- **A run rollover no longer strands the device snapshot.** `deviceOverrides` lives in the run,
|
|
238
|
+
and `rolloverReason` *seals* runs on a device/session change or idle timeout — so the snapshot
|
|
239
|
+
used to leave with the archive while `vk device reset`, which reads the active run, reported
|
|
240
|
+
"nothing to restore" and left the device modified. Found the hard way: two workspaces driving
|
|
241
|
+
different devices from one working directory ping-ponged the active run and left a phone dark
|
|
242
|
+
and rotated. A same-device rollover now carries the overrides **forward**; a device-change
|
|
243
|
+
rollover warns with the exact `vk device set … --device <serial>` needed to undo it, since a
|
|
244
|
+
process pointed at the new device cannot drive the old one and restoring one device's values
|
|
245
|
+
onto another would be worse than leaving it.
|
|
246
|
+
- `RunState` gained `deviceOverrides`, and `device` joined `RECORDABLE` — a report showing
|
|
247
|
+
"checkout failed" without "we cut the network two steps earlier" points the reader at the
|
|
248
|
+
wrong culprit. `stepName` renders the whole assignment list (`device set dark=on`).
|
|
249
|
+
- `exec.ts` gained `sleepSync`: the `Driver` interface is entirely synchronous, so a readback
|
|
250
|
+
poll cannot await a timer. Built on `Atomics.wait`, keeping the zero-runtime-dependency rule.
|
|
251
|
+
|
|
252
|
+
### Fixed
|
|
253
|
+
- Nothing user-visible yet (this release adds a new capability), but two traps are handled
|
|
254
|
+
rather than inherited: **`airplane=on` is verified by effect, not by its flag** — Android
|
|
255
|
+
remembers a user who re-enabled wifi during a previous flight
|
|
256
|
+
(`airplane_mode_toggleable_radios`), so `airplane-mode enable` can leave wifi *up*; verikun
|
|
257
|
+
reads that list off the device, confirms each radio it names actually went down, and forces
|
|
258
|
+
any survivor — because reporting "offline" while the app is still online would make an
|
|
259
|
+
offline test pass for the wrong reason. And **`airplane=on` is refused over a wireless adb
|
|
260
|
+
link** (exit 2, `--allow-wireless` overrides), since it would cut the very connection
|
|
261
|
+
carrying the next command, with no way to turn it back on remotely.
|
|
262
|
+
|
|
263
|
+
## [0.18.1] - 2026-08-07
|
|
264
|
+
|
|
265
|
+
### Fixed
|
|
266
|
+
- **A `vk ai` run the engine failed is no longer archived as a GREEN report.** A test that
|
|
267
|
+
failed because a **control node** failed — a `repeat` that exhausted without its target
|
|
268
|
+
ever appearing, a `when` that matched no branch with no `else` — recorded no step, and
|
|
269
|
+
both `report.xml` and `report.html` derived their verdict purely from the step tally. So
|
|
270
|
+
the JUnit XML declared `failures="0"` for a failed test, in exactly the format CI trusts
|
|
271
|
+
automatically, and the HTML showed all-PASS while the suite summary said FAIL. The same
|
|
272
|
+
hole covered `read` failures, a repair give-up, and the budget/timeout aborts (which the
|
|
273
|
+
engine reports as a bare flag with no reason attached at all).
|
|
274
|
+
|
|
275
|
+
`Recorder.recordTerminalFailure` now records the engine's verdict before the archive: the
|
|
276
|
+
run gains a top-level `failure` (`where` + reason, the same text the suite prints), and a
|
|
277
|
+
synthetic failed step is appended — but **only when no step is already red**, so a leaf
|
|
278
|
+
failure that already has its own testcase is not counted twice. `vk run archive`'s exit
|
|
279
|
+
code and `vk suite`'s `failedSteps` follow from that with no further change.
|
|
280
|
+
- **Failure evidence for a failure no command produced.** `ExecBackend.captureFailure`
|
|
281
|
+
grabs a screenshot + the UI hierarchy at the moment the engine gives up, attached to the
|
|
282
|
+
recorded step exactly like a failing command's `step-N-fail.png`. Previously the archive
|
|
283
|
+
of a control-node failure held no trace of the screen at all. Over `--server` this
|
|
284
|
+
degrades honestly to the hierarchy only — the server has no screenshot route.
|
|
285
|
+
- **The report writer no longer infers pass/fail from the step tally alone.** `runFailure()`
|
|
286
|
+
takes the run-level verdict (`failure`, falling back to `ai.ok`); if a failed run somehow
|
|
287
|
+
still carries no red step, both renderers emit one synthetic failing testcase rather than
|
|
288
|
+
reporting green. `report.html` also states the verdict in a banner at the top, since that
|
|
289
|
+
page is where a human looks first.
|
|
290
|
+
|
|
291
|
+
## [0.18.0] - 2026-08-06
|
|
292
|
+
|
|
293
|
+
### Added
|
|
294
|
+
- **Archive-time device log capture by default** ([#56]) — `vk run archive` (and
|
|
295
|
+
rollover / `vk ai` seals) now write a bounded, session-scoped logcat dump to
|
|
296
|
+
`artifacts/logcat.txt` without requiring an explicit `vk log` step. Opt out on
|
|
297
|
+
green runs with `--no-logs` or `VERIKUN_NO_LOGS`; failed runs still capture.
|
|
298
|
+
Capture is best-effort and never blocks sealing the report. Full dump linked from
|
|
299
|
+
`report.html` meta; when the run launched an app, an app-scoped dump is also written
|
|
300
|
+
to `artifacts/logcat-app.txt` and embedded in a bottom accordion. Noted in `report.xml`.
|
|
301
|
+
- **`logStart` over `--server`** ([#56]) — `/v1/exec` returns the device-clock
|
|
302
|
+
marker and `/v1/logs` serves archive-time dumps, so remote CI archives scope
|
|
303
|
+
logs the same way a local run does.
|
|
304
|
+
|
|
305
|
+
## [0.17.0] - 2026-08-06
|
|
306
|
+
|
|
307
|
+
### Added
|
|
308
|
+
- **`vk suite --retries N`** ([#43]) — re-run a failed test up to N times so a
|
|
309
|
+
transient flake can recover the suite instead of reding CI. A later pass exits
|
|
310
|
+
`0` and surfaces a warning (`flaky` + prior `attempts` on the test row, suite
|
|
311
|
+
`warnings` in the manifest/HTML); failed-attempt archives stay linked. Default
|
|
312
|
+
`0` keeps today's gate.
|
|
313
|
+
|
|
314
|
+
The bias is deliberately toward retrying: an attempt costs one test, while giving
|
|
315
|
+
up costs the whole suite plus a human rerunning it. So an **environment break is
|
|
316
|
+
retried too** — a `vk server` connection dropping or a device re-enumerating
|
|
317
|
+
outlives the two-probe health window, and with attempts left the suite waits
|
|
318
|
+
(short backoff, one `warnings` entry per blip) rather than aborting. Only two
|
|
319
|
+
failures are never retried, because a rerun provably cannot change them: a **budget
|
|
320
|
+
abort** (each attempt gets its own ceiling) and a **usage error** (exit `2` — an
|
|
321
|
+
unreadable test file, a payload the server refuses). Once the attempts are gone, a
|
|
322
|
+
still-broken environment aborts with exit `3` exactly as before.
|
|
323
|
+
|
|
324
|
+
[#43]: https://github.com/ddikman/verikun/issues/43
|
|
325
|
+
## [0.16.0] - 2026-08-05
|
|
326
|
+
|
|
327
|
+
### Added
|
|
328
|
+
- **`--selected`, `--checked`, `--focused` selector modifiers, each with a `--not-` form**
|
|
329
|
+
([#38]) — `--not-selected`, …, joining `--enabled`, which also gains `--not-enabled`. `selected`
|
|
330
|
+
and `checked` were already parsed into `Element` and printed by `vk ui`; there was simply
|
|
331
|
+
no way to match on them. Unset means *don't care*; the filter is applied to the candidate
|
|
332
|
+
pool **before** the healing tiers, so a state-matching exact hit can never shadow a
|
|
333
|
+
state-matching partial one. Combining `--x` with `--not-x` is a usage error (exit 2).
|
|
334
|
+
|
|
335
|
+
The negative form is the load-bearing half. A segmented control whose options share one
|
|
336
|
+
handler *flips* on any tap, and its default is content-driven — so "tap the option I
|
|
337
|
+
want" lands on the other one whenever it was already chosen. Measured on the fixture:
|
|
338
|
+
`vk tap @vk_mode_video` exits 0, prints `tapped … tap,selected`, and leaves the app in
|
|
339
|
+
photo mode. The flow completes, nothing fails, and the run exercised the opposite mode —
|
|
340
|
+
worse than a red test. `if-present "id:x --not-selected" { tap id:x }` makes the
|
|
341
|
+
already-correct case a no-op instead.
|
|
342
|
+
- **A selector string may carry its own state modifiers** — `"id:mode_video --not-selected"`.
|
|
343
|
+
Control nodes (`if-present`, `when`, `repeat`, `while-present`, `read`) hold a bare
|
|
344
|
+
`selector: string` with nowhere to put a flag, and a guard is exactly where the toggle
|
|
345
|
+
case needs one. Stripped only at the end of the string and only after whitespace, so
|
|
346
|
+
`text:--selected` and `text:a --selected b` stay plain values; `Selector.raw` keeps the
|
|
347
|
+
original so errors and reports echo what was written. `swipe --on` and every command
|
|
348
|
+
inherit the same parser.
|
|
349
|
+
- **The `vk ai` grammar teaches all of it**, including the shared-handler toggle rule, so a
|
|
350
|
+
compiled plan can express the guard. Verified end to end: the compiler emitted
|
|
351
|
+
`{"type":"if-present","selector":"id:vk_mode_video --not-selected", …}` unprompted and
|
|
352
|
+
the plan replayed model-free at `$0`.
|
|
353
|
+
- **A fourth fixture screen, `@vk_state`** (`example/flutter-app`) — a mode picker whose two
|
|
354
|
+
options share one `_toggleMode()`, plus a field that takes real input focus. It is the
|
|
355
|
+
instrument the facts below were measured with, and the device suite drives it.
|
|
356
|
+
|
|
357
|
+
### Fixed
|
|
358
|
+
- **`vk tap --enabled @submit` swallowed its selector.** `enabled` was missing from the
|
|
359
|
+
argv parser's `BOOLEAN` set, and a non-boolean flag consumes the next token — so the
|
|
360
|
+
selector became the flag's value and the command died with "Missing selector". Only the
|
|
361
|
+
trailing form worked. All eight new modifier names are registered too.
|
|
362
|
+
- **`swipe --on <selector>` silently ignored `--index` and `--enabled`.** It built its
|
|
363
|
+
selector directly instead of going through `buildSelector`; it now honours every
|
|
364
|
+
modifier like the other selector commands.
|
|
365
|
+
|
|
366
|
+
### Platform notes
|
|
367
|
+
- **`--selected` / `--focused` are Android-only, and say so.** Measured against the fixture
|
|
368
|
+
on an iPhone 17 Pro simulator: a selected and an unselected segment come back identical
|
|
369
|
+
apart from label and frame. `idb ui describe-all --json` has no such key in its schema at
|
|
370
|
+
all — no accessibility traits — so no app can supply it and there is nothing to derive.
|
|
371
|
+
Using them with `--ios` therefore exits **3** with a named reason rather than matching
|
|
372
|
+
nothing: a filter that can only ever match zero elements burns the full auto-wait window
|
|
373
|
+
and then reports "No element matched selector", which is untrue about the screen and is
|
|
374
|
+
precisely the false signal these modifiers were added to prevent. Same honest degrade as
|
|
375
|
+
`clearApp` and `currentApp`. `--enabled` and `--checked` work on both platforms
|
|
376
|
+
(`checked` is derived on iOS from the element type plus `AXValue`).
|
|
377
|
+
|
|
378
|
+
[#38]: https://github.com/ddikman/verikun/issues/38
|
|
379
|
+
|
|
380
|
+
## [0.15.0] - 2026-08-05
|
|
381
|
+
|
|
382
|
+
### Fixed
|
|
383
|
+
- **`tap`/`text` no longer press a point that does not reach their target** ([#42]).
|
|
384
|
+
An element's centre is not always on the element: a row scrolled past the edge of its
|
|
385
|
+
list arrives with bounds already clipped to the display, and a sticky bottom bar is
|
|
386
|
+
drawn across the middle of whatever is under it. Pressing that point hit the *other*
|
|
387
|
+
control — and the step reported success, so the run continued from the wrong place and
|
|
388
|
+
failed several steps later on an unrelated symptom. Reproduced on a Pixel 3a: pressing
|
|
389
|
+
a row's own centre recorded the sticky bar's button, not the row.
|
|
390
|
+
|
|
391
|
+
### Added
|
|
392
|
+
- **Auto scroll-into-view.** `tap` and `text` now bring their target into the clear
|
|
393
|
+
before acting — into its scroll container, and out from under anything painted over
|
|
394
|
+
it — then act, appending `(scrolled into view: N swipes)` to the confirmation. "Scroll
|
|
395
|
+
down to X and tap it" is just `vk tap X`; the `repeat until X { swipe up }` idiom is no
|
|
396
|
+
longer needed to reach something below the fold (the `vk ai` grammar now says so).
|
|
397
|
+
Where the target cannot be reached at all, the action **fails with exit 1** instead of
|
|
398
|
+
tapping blind coordinates. `--no-scroll` opts out.
|
|
399
|
+
- **`offscreen` marker** on elements with no pixel on screen, in `vk ui`'s output and
|
|
400
|
+
`--json`. Nothing is hidden — inspection commands (`ui`/`find`/`assert`) neither scroll
|
|
401
|
+
nor filter; only actions move the screen.
|
|
402
|
+
- **`Driver.viewport()`** — `screenSize()` corrected for the dump's rotation, memoized
|
|
403
|
+
per driver instance. Every unknown degrades to "everything is visible", so a device
|
|
404
|
+
whose screen size cannot be read behaves exactly as before.
|
|
405
|
+
- A **scroll screen** in the Flutter fixture (`example/flutter-app`) where every row
|
|
406
|
+
records which control actually fired, plus e2e coverage that pins the wrong-tap
|
|
407
|
+
reproduction, and two new measured facts in its README: Android's dumper hides
|
|
408
|
+
off-screen nodes and clips the rest, and a fast swipe can take the app off the screen.
|
|
409
|
+
|
|
410
|
+
[#42]: https://github.com/ddikman/verikun/issues/42
|
|
411
|
+
## [0.14.0] - 2026-08-03
|
|
412
|
+
|
|
413
|
+
### Added
|
|
414
|
+
- **`suggest-verikun-improvement` skill.** When *verikun itself* is the friction — a model
|
|
415
|
+
heal on a **cached** replay (the $0 replay that still woke the model), a repair give-up,
|
|
416
|
+
or a recurring gotcha — the main `verikun` skill hands off to this new skill, which drafts
|
|
417
|
+
a light, **TL;DR-first** improvement suggestion for `ddikman/verikun`. It is
|
|
418
|
+
**draft-first** (the user reviews before anything is submitted) and **aggressively redacts**
|
|
419
|
+
every app-under-test specific (package, on-screen text, selector values, test prose, logs,
|
|
420
|
+
screenshots) so no client code or logic can leak. De-dup is by a generalised *category*
|
|
421
|
+
fingerprint, so one issue tracks each verikun weakness class rather than one per app or
|
|
422
|
+
control.
|
|
423
|
+
- **`cached` in `vk ai --json`.** The structured summary now reports whether the plan was a
|
|
424
|
+
cached replay (`true`) or a fresh compile (`false`), so a heal on a cached replay — the
|
|
425
|
+
signal the skill keys on — is detectable from the JSON instead of by parsing the
|
|
426
|
+
`[ai] plan cache hit` stderr line.
|
|
427
|
+
|
|
428
|
+
## [0.13.0] - 2026-08-03
|
|
429
|
+
|
|
430
|
+
### Added
|
|
431
|
+
- **`vk ai --model gpt-4.1` — a cheaper alternative to the default `claude-sonnet-4-6`.** $2/$8 per
|
|
432
|
+
1M against sonnet's $3/$15, priced in the bundled table like every other model. It reuses the
|
|
433
|
+
existing `OpenAiProvider` (Chat Completions + strict `json_schema` Structured Outputs, which
|
|
434
|
+
gpt-4.1 supports) and `OPENAI_API_KEY`, so it is a `MODELS` row rather than a new backend.
|
|
435
|
+
- `Price.cacheReadMult` — an optional per-model override for the cache-read multiplier. gpt-4.1
|
|
436
|
+
bills cache reads at **0.25x** input ($0.50/1M), not the 0.1x every model in the table until now
|
|
437
|
+
shared, and that table is what the `--max-cost-usd` gate meters. A `--cost-override` carries no
|
|
438
|
+
multiplier and so still assumes 0.1x.
|
|
439
|
+
|
|
440
|
+
### Fixed
|
|
441
|
+
- `reasoning_effort` is now sent only to OpenAI models that accept it (a `REASONING_MODELS`
|
|
442
|
+
allowlist, mirroring `claude.ts`'s `EFFORT_MODELS`). gpt-4.1 is the registry's first
|
|
443
|
+
**non-reasoning** model and OpenAI rejects the parameter for it with a `400` — which the retry
|
|
444
|
+
loop deliberately does not retry — so unguarded, `--model gpt-4.1 --effort high` would have died
|
|
445
|
+
as a usage error (exit `2`) before reaching the device. An allowlist rather than a denylist: an
|
|
446
|
+
unlisted model quietly forgoes effort instead of failing the run.
|
|
447
|
+
|
|
448
|
+
## [0.12.0] - 2026-08-03
|
|
449
|
+
|
|
450
|
+
### Added
|
|
451
|
+
- **Toolchain preflight** — device setup is now verified before tests run and will fail the suite instead of test-by-test.
|
|
452
|
+
It will verify that idb and/or adb are available and that there are devices that can be used.
|
|
453
|
+
- **An aborted suite says which tests never ran** — `index.json` gains an `aborted` block
|
|
454
|
+
and `index.html` a banner. Skipped tests are not counted as failures.
|
|
455
|
+
|
|
456
|
+
### Fixed
|
|
457
|
+
- **`vk ai` and `vk suite` exit `3` (was `1`) when the environment is what failed**, so a
|
|
458
|
+
broken machine no longer looks like a failing app. Restores the documented exit-code contract.
|
|
459
|
+
- **A control-flow guard no longer reads an unreadable screen as "absent".** It used to skip
|
|
460
|
+
its body, so a guard-heavy plan could pass having executed nothing.
|
|
461
|
+
- **No more duplicate `could not capture failure …` noise** when the toolchain is itself why
|
|
462
|
+
the step failed. Evidence capture is still attempted, just not narrated twice.
|
|
463
|
+
|
|
464
|
+
### Known gap
|
|
465
|
+
- A failing `screenshot` step still stays green even when a missing tool caused it: a genuine
|
|
466
|
+
screencap hiccup looks identical, so telling them apart needs a separate error type.
|
|
467
|
+
|
|
468
|
+
## [0.11.0] - 2026-07-31
|
|
469
|
+
|
|
470
|
+
### Added
|
|
471
|
+
- **`--enabled` selector modifier** — match only a control that is *actionable right now*
|
|
472
|
+
(`enabled` plus, on Android, `clickable`/`longClickable`), and with auto-wait, block until it
|
|
473
|
+
becomes so. A Submit/Check button that the app disables until a form validates is **present**
|
|
474
|
+
long before it is **usable**, so a presence match taps a dead control, nothing happens, and the
|
|
475
|
+
failure surfaces several steps later as a confusing timeout on whatever should have come next.
|
|
476
|
+
Found by porting a real flow whose Maestro original needed the same guard (`enabled: true`).
|
|
477
|
+
The filter is applied to the candidate pool *before* the healing tiers, so a disabled exact
|
|
478
|
+
match cannot shadow an enabled partial one.
|
|
479
|
+
- **`vk ai` plans can branch, loop over an unknown-length list, and carry values — the fix for
|
|
480
|
+
compile-from-one-playthrough (#33).** Four additions to the plan IR, all evaluated by the
|
|
481
|
+
engine at replay, so a green run still costs **$0** and wakes no model:
|
|
482
|
+
- **`when`** — ordered n-way dispatch: the first branch whose selector is on screen runs, and
|
|
483
|
+
only it. For "this screen is one of N kinds, handle whichever it is". No match and no `else`
|
|
484
|
+
is a **failure**, deliberately the opposite of `if-present` ("this may or may not be there").
|
|
485
|
+
A silent skip inside a loop would spin to the cap doing nothing and report green.
|
|
486
|
+
- **One level of control nesting**, so `repeat { when { … } }` — the loop-that-branches shape
|
|
487
|
+
a dynamic flow needs — is finally expressible. `ir.ts` said *"a real flow that needs deeper
|
|
488
|
+
nesting is the trigger to revisit"*; this was that flow.
|
|
489
|
+
- **`while-present`** with an optional `bind` counter: repeats while a selector is present,
|
|
490
|
+
exposing the index as `{{ctx.<name>}}`. Walks an index-addressed list whose length is not
|
|
491
|
+
knowable at compile time ("tap each pair until they are all matched"). It may nest one level
|
|
492
|
+
deeper than the branching nodes, since its own body is leaves.
|
|
493
|
+
- **`read`** — capture `text`/`desc`/`id`/`idShort` off the live tree into `{{ctx.<name>}}`,
|
|
494
|
+
for the case where a test must act on a value it cannot know in advance (an app that marks
|
|
495
|
+
the correct answer in the semantic tree, and the test must then *type* it).
|
|
496
|
+
- **Placeholders** in any positional, flag value, or control-node selector: `{{ctx.NAME}}`,
|
|
497
|
+
`{{env.NAME}}`, `{{uuid}}`, `{{timestamp}}`, `{{run_id}}`. `{{uuid}}`/`{{timestamp}}` are
|
|
498
|
+
generated **once per run** and memoized, so a signup uses the same address in the email field,
|
|
499
|
+
the confirm field and a later assert — and a different one next run, which is what stops a
|
|
500
|
+
cached plan from colliding with the account its previous run created. The store is per
|
|
501
|
+
`runPlan` call, not per process, so two tests in one `vk suite` do not share a value.
|
|
502
|
+
An unset `{{ctx.*}}` or `{{env.*}}` **fails the step** rather than substituting an empty
|
|
503
|
+
string. Placeholders live in the plan, never in the prose, so the cache key is unchanged.
|
|
504
|
+
This is deliberately template substitution, not an expression language — there is no `eval`,
|
|
505
|
+
no `vm`, and nothing to sandbox.
|
|
506
|
+
|
|
507
|
+
### Changed
|
|
508
|
+
- **A `repeat` that stops without ever seeing its selector now FAILS.** Both exits — cap
|
|
509
|
+
exhausted and the structural no-progress bail — used to return success, so a loop whose body
|
|
510
|
+
did nothing reported green. That is the reachable false green once bodies can branch (a `when`
|
|
511
|
+
that matches nothing does nothing, the loop spins, the run passes). A post-loop confirmation
|
|
512
|
+
check runs before failing, so a target that appears after the final iteration is not a false
|
|
513
|
+
red. **This can turn a previously-green scroll-until red if it never actually found its row.**
|
|
514
|
+
- `validateNode` rejects an empty control body (`when`'s `else: []` is the one exception — the
|
|
515
|
+
explicit "match nothing, do nothing" opt-in), and tolerates `null` for optional fields, which
|
|
516
|
+
is how OpenAI/codex strict mode encodes an absent `else`/`bind`.
|
|
517
|
+
|
|
518
|
+
### Fixed
|
|
519
|
+
- **`vk ai`: the loop no-progress guard was blind to `content-desc`, so it aborted healthy loops
|
|
520
|
+
on Flutter apps.** `structuralHash` (`src/agent/engine.ts`) fingerprinted the screen as
|
|
521
|
+
`idShort|text|type`. Flutter maps `Semantics(label:)` to Android's `contentDescription` — i.e.
|
|
522
|
+
to `desc`, never to `text` — so on a Flutter app every content-bearing node has an empty `text`
|
|
523
|
+
and the fingerprint collapsed to `id||type` for the whole screen. Measured on a live screen: 14
|
|
524
|
+
elements, **0** with `text`, 8 with `desc`. Two entirely different questions hashed
|
|
525
|
+
byte-identically, and a loop answering a different one each iteration was declared stalled and
|
|
526
|
+
failed. Now `id|text|desc|type` — and the full `id` rather than `idShort` (the suffix after the
|
|
527
|
+
last `/`), so nodes from different namespaces cannot collide in what is meant to be a
|
|
528
|
+
fingerprint. Presented as flakiness rather than a hard failure, since it only fired when the app
|
|
529
|
+
happened to serve several same-shaped screens in a row.
|
|
530
|
+
- **`assert --text` now matches `content-desc` as well as `text`.** The same blindness in a second
|
|
531
|
+
place: `evalAssert` compared `m.text` only, so on a Flutter app `assert @x --text "Welcome"`
|
|
532
|
+
could never pass — while `text:Welcome` as a *selector* resolved fine, because the selector layer
|
|
533
|
+
already falls back to `desc`. The two now agree. Strictly widening: it can only turn a false
|
|
534
|
+
negative into a pass.
|
|
535
|
+
- **`vk ai`: an `if-present` guard now waits for its selector before deciding the optional UI is
|
|
536
|
+
absent.** `present()` (`src/agent/engine.ts`) took a single `uiautomator` dump and trusted a
|
|
537
|
+
successful-but-empty result — its retry loop only re-fired when the dump *threw*. Every
|
|
538
|
+
selector-resolving leaf command auto-waits ~5s, so a guard was strictly **less patient than a
|
|
539
|
+
bare `tap`**, and an interstitial that animated in a few hundred ms after a transition was
|
|
540
|
+
missed by the very construct meant to catch it. This is the first of the fixes for the
|
|
541
|
+
compile-from-one-playthrough problem (#33) — optional interstitials were one of its four
|
|
542
|
+
variation classes.
|
|
543
|
+
- The window is **not** a pure wall-clock box. A UI dump measured ~2.4s on an emulator and can be
|
|
544
|
+
~10x faster on a physical device, so a time-only budget gives a fast phone a dozen looks at the
|
|
545
|
+
screen and a slow emulator none — the first implementation of this fix was a silent no-op on
|
|
546
|
+
exactly the devices that needed it. A non-zero window therefore guarantees **at least two
|
|
547
|
+
looks**, and keeps polling while time remains. Cost of an absent guard is about one extra dump
|
|
548
|
+
(~2-3.5s on the reference emulator, proportionally less on real hardware).
|
|
549
|
+
- A **loop's own exit check never pays the window** (`settleMs = 0`). That guard is absent on
|
|
550
|
+
every iteration by construction — that is what makes it a loop — so inheriting the window would
|
|
551
|
+
burn `cap ×` it (~37s at cap 25) to discover something already expected.
|
|
552
|
+
- The transient-dump retry is unchanged and **orthogonal** to the new window: a dump that throws
|
|
553
|
+
is still retried once even with the window closed, so a flaky `uiautomator` call never reads as
|
|
554
|
+
"absent" and skips a body that should have run.
|
|
555
|
+
|
|
556
|
+
### Added
|
|
557
|
+
- **`VERIKUN_GUARD_SETTLE_MS`** — tune the `vk ai` guard settle window without a rebuild (`0`
|
|
558
|
+
restores the previous single-shot probe). The right value is device-dependent, so it is meant
|
|
559
|
+
to be measured against a real app rather than guessed.
|
|
560
|
+
|
|
561
|
+
|
|
562
|
+
## [0.10.0] - 2026-07-26
|
|
563
|
+
|
|
564
|
+
### Added
|
|
565
|
+
- **`vk ai --model cursor-cli` — compile/repair off a logged-in `cursor-agent`, no API key.**
|
|
566
|
+
The second CLI-agent backend, stacking on the same `CliProvider` class 0.9.0 introduced: it is
|
|
567
|
+
a new `CliAgentSpec` (`CURSOR_SPEC`), not a new provider class. Anyone with a Cursor
|
|
568
|
+
subscription and `cursor-agent login` can now run `vk ai` / `vk suite` without setting
|
|
569
|
+
`ANTHROPIC_API_KEY` or `OPENAI_API_KEY`. Like `codex-cli` it is billed to the subscription, so
|
|
570
|
+
its cost line reads `$0` and `--max-cost-usd` / `--cost-override` are inert.
|
|
571
|
+
- Unlike `codex`, `cursor-agent` has **no JSON-schema flag**, so the plan schema is injected into
|
|
572
|
+
the prompt and `extractJson` peels the object back out of the `--output-format json` envelope;
|
|
573
|
+
`parsePlan` / `validateNode` remains the execution trust boundary either way. The call runs
|
|
574
|
+
`--mode ask` (read-only) with `--workspace` pointed at a scratch dir, so it never sees — let
|
|
575
|
+
alone touches — your working tree, and is never given `--force` / `--yolo`. A `cursor-agent`
|
|
576
|
+
run that reports `is_error` while still exiting `0` is surfaced as an environment error (exit
|
|
577
|
+
`3`) rather than a confusing "did not return parseable JSON".
|
|
578
|
+
|
|
579
|
+
### Changed
|
|
580
|
+
- `providerAvailable` / `providerRequirement` / `makeProvider` (`cli.ts`) now look CLI backends up
|
|
581
|
+
in one `CLI_SPECS` table instead of each carrying a per-CLI `switch` arm, and the "not found on
|
|
582
|
+
PATH" error reuses the spec's own `loginHint` rather than restating it. Adding a third agent CLI
|
|
583
|
+
is a spec plus a table entry.
|
|
584
|
+
- Corrected the `AgentProvider` docs in `CLAUDE.md` and `src/agent/provider.ts`, which still
|
|
585
|
+
described a single v1 Claude implementation and never mentioned the OpenAI or CLI backends.
|
|
586
|
+
|
|
587
|
+
## [0.9.0] - 2026-07-21
|
|
588
|
+
|
|
589
|
+
### Added
|
|
590
|
+
- **`vk ai --model codex-cli` — drive the compiler/repair off a logged-in `codex` CLI, no
|
|
591
|
+
API key.** A third provider backend (`CliProvider`, `src/agent/cli-provider.ts`) shells out
|
|
592
|
+
to an already-authenticated coding-agent CLI instead of calling an HTTP API, so anyone with
|
|
593
|
+
a ChatGPT subscription and `codex login` can run `vk ai` / `vk suite` without setting
|
|
594
|
+
`ANTHROPIC_API_KEY` or `OPENAI_API_KEY`. It runs `codex exec` read-only in a neutral temp
|
|
595
|
+
dir (never touching the working tree) and uses codex's native `--output-schema` to constrain
|
|
596
|
+
the plan JSON; `parsePlan`/`validateNode` remains the trust boundary regardless of provider.
|
|
597
|
+
The provider seam (`AgentProvider`) and the shared prompt/grammar are unchanged — it's one
|
|
598
|
+
more class behind the same `providerFor(model)` routing. (The `cursor-cli` backend will stack
|
|
599
|
+
on the same class next.)
|
|
600
|
+
|
|
601
|
+
### Changed
|
|
602
|
+
- Provider availability is now per-provider: HTTP providers still need their API key in the
|
|
603
|
+
environment, while a CLI provider (`codex-cli`) is available when its binary is on PATH. The
|
|
604
|
+
`vk ai`/`vk suite` preflight errors reflect this (e.g. "the `codex` CLI was not found on PATH
|
|
605
|
+
— run `codex login`") instead of only naming an env var.
|
|
606
|
+
- For `codex-cli`, spend is billed to the user's subscription rather than per token, so cost is
|
|
607
|
+
reported as `$0` and `--max-cost-usd` / `--cost-override` are inert no-ops; a run is bounded
|
|
608
|
+
by `maxRepairs` and `--timeout` instead. Plans stay provider-agnostic in the cache, so a plan
|
|
609
|
+
compiled by any model replays for free under `--model codex-cli` (use `--recompile` to force
|
|
610
|
+
a fresh compile when comparing providers).
|
|
611
|
+
|
|
612
|
+
## [0.8.1] - 2026-07-21
|
|
613
|
+
|
|
614
|
+
### Changed
|
|
615
|
+
- `vk devices` now prints an aligned, headed table — a header row plus columns that
|
|
616
|
+
line up across Android/iOS rows, with empty optional columns (MODEL/PRODUCT/NOTE)
|
|
617
|
+
omitted. Previously rows were tab-joined and dropped empty cells, so a device
|
|
618
|
+
missing a model/product slid its remaining cells out of alignment. `--json` output
|
|
619
|
+
is unchanged.
|
|
620
|
+
|
|
621
|
+
## [0.8.0] - 2026-07-15
|
|
622
|
+
|
|
623
|
+
### Added
|
|
624
|
+
- **Automatic review screenshots in `vk ai` / `vk suite`.** The compiler now inserts
|
|
625
|
+
`screenshot` steps around screen transitions and inside `if-present`/`repeat` bodies,
|
|
626
|
+
so the archived HTML/JUnit report carries a before/after visual trail for post-run
|
|
627
|
+
review. They are dumped for humans, never read back by the model (no token cost on
|
|
628
|
+
replay), and are best-effort — a capture that fails is logged and skipped, never a
|
|
629
|
+
test failure. Guidance for driving `vk` by hand is in SKILL.md / README (a screenshot
|
|
630
|
+
taken as report evidence and never read back is free — only *reading* an image into an
|
|
631
|
+
agent's context costs tokens).
|
|
632
|
+
|
|
633
|
+
### Changed
|
|
634
|
+
- `vk ai` / `vk suite` reports and JUnit now contain more `<testcase>` steps per run
|
|
635
|
+
(the auto-inserted screenshots), so step/testcase counts are higher than in 0.7.0.
|
|
636
|
+
- The `vk ai` engine treats a `screenshot`/`shot` step that fails to capture as
|
|
637
|
+
best-effort: it logs the failure and continues instead of failing the run.
|
|
638
|
+
|
|
639
|
+
## [0.7.0] - 2026-07-14
|
|
640
|
+
|
|
641
|
+
### Added
|
|
642
|
+
- **`vk suite <dir>`** — run a directory of natural-language tests (`*.md`,
|
|
643
|
+
lexicographic order, `README.md` skipped) as one sequential suite: app data is
|
|
644
|
+
reset between tests when `--app <id>` is given (iOS degrades to a force-stop),
|
|
645
|
+
each test runs through the `vk ai` engine against one shared backend, and the
|
|
646
|
+
suite writes an overview to `./.verikun/suites/<id>/` — `index.json` (a stable,
|
|
647
|
+
`schemaVersion`ed manifest with per-test pass/fail, steps, repairs, cost, and
|
|
648
|
+
duration) plus `index.html` linking every test's archived report. Exits 1 when
|
|
649
|
+
any test failed, so the command doubles as the CI gate. Reporting providers
|
|
650
|
+
compose over the manifest as CI steps (upload-artifact, rclone, `aws s3`) —
|
|
651
|
+
no in-core upload plugins.
|
|
652
|
+
- **`vk server`** — expose the locally-connected device/simulator to remote
|
|
653
|
+
verikun clients over HTTP+JSON (Node built-ins only). Only verikun's validated
|
|
654
|
+
action grammar is executable (every `/v1/exec` request passes the same
|
|
655
|
+
`validateNode` gate that guards `vk ai` model repairs — never `ui`/`log`, never
|
|
656
|
+
a shell), the device/platform are fixed at startup (client flags can't repoint
|
|
657
|
+
them), and auth is mandatory: a bearer key via `--auth-key` /
|
|
658
|
+
`VERIKUN_SERVER_AUTH_KEY` (auto-generated and printed if omitted), compared with
|
|
659
|
+
`crypto.timingSafeEqual`; `--allow-unsafe-anonymous` opts out loudly for
|
|
660
|
+
networks that are themselves the boundary (e.g. Tailscale). Binds `127.0.0.1`
|
|
661
|
+
by default (`--bind`/`--port` to expose); one run-token holds the device lock
|
|
662
|
+
at a time (409 otherwise, idle locks are taken over); `--allow-install` enables
|
|
663
|
+
the build-upload endpoint.
|
|
664
|
+
- **`--server <url>` remote mode for `vk ai` / `vk suite` / `vk install`** (or
|
|
665
|
+
`VERIKUN_SERVER`, with `--auth-key` / `VERIKUN_SERVER_AUTH_KEY`): the whole
|
|
666
|
+
engine — compile, plan cache, repair, run recording, reports, cost budget —
|
|
667
|
+
runs on the caller (e.g. a disposable CI runner with the Anthropic key), while
|
|
668
|
+
only validated leaf commands cross the network, one round-trip per command
|
|
669
|
+
(auto-wait polling stays server-side). Each step's detail (selector, heal tier,
|
|
670
|
+
resolved element, failure screenshot + hierarchy) is spliced back into the
|
|
671
|
+
local run, so a remote run's report is identical to a local one's.
|
|
672
|
+
- **`vk install <app.apk|.ipa>`** — install a build on the device (`adb install
|
|
673
|
+
-r` / `idb install`). With `--server`, the file is streamed to the server
|
|
674
|
+
(which must run `--allow-install`) with sha256 integrity verification; the
|
|
675
|
+
remote path accepts single-file `.apk`/`.ipa` only.
|
|
676
|
+
- **`.github/workflows/suite.yml`** — a reference GitHub Actions recipe: a
|
|
677
|
+
throwaway `ubuntu-latest` runner builds verikun, installs the app build on the
|
|
678
|
+
remote device via `vk install --server`, runs `vk suite --server` against it,
|
|
679
|
+
uploads `.verikun/suites` + `.verikun/runs` as artifacts (with commented rclone
|
|
680
|
+
/ S3 provider examples over the manifest), and fails the job on any failed test.
|
|
681
|
+
|
|
682
|
+
## [0.6.0] - 2026-07-10
|
|
683
|
+
|
|
684
|
+
### Added
|
|
685
|
+
- **OpenAI models for `vk ai`.** `--model` now switches provider as well as model:
|
|
686
|
+
alongside the Claude models it accepts `gpt-5.4-mini`, `gpt-5.4`, and `gpt-5.5`, served
|
|
687
|
+
by a new `OpenAiProvider` (`src/agent/openai.ts`) that calls OpenAI's Chat Completions
|
|
688
|
+
API over Node's built-in `fetch` — no SDK, so the zero-runtime-dependency rule still
|
|
689
|
+
holds. OpenAI models read `OPENAI_API_KEY`; Claude models continue to read
|
|
690
|
+
`ANTHROPIC_API_KEY`. The compile-once / replay-model-free engine, plan cache, and cost
|
|
691
|
+
budget are unchanged — the provider is chosen behind the existing `AgentProvider` seam.
|
|
692
|
+
Because the endpoint is the de-facto OpenAI-compatible shape, the same provider can
|
|
693
|
+
later target Groq/xAI/Together/etc. by base URL.
|
|
694
|
+
|
|
695
|
+
## [0.5.0] - 2026-07-10
|
|
696
|
+
|
|
697
|
+
### Added
|
|
698
|
+
- **iOS simulator + device support at parity with Android**, via Facebook's
|
|
699
|
+
[`idb`](https://github.com/facebook/idb). `vk --ios` now drives the full loop —
|
|
700
|
+
`ui`/`find`, `tap`, `text`/`type`, `swipe`, `key`, `assert`, `wait`, plus the
|
|
701
|
+
existing `screenshot`/`launch`/`stop`, and therefore `vk batch`, `vk ai`, and
|
|
702
|
+
the JUnit/HTML test-run reports. The hierarchy comes from `idb ui describe-all`
|
|
703
|
+
(parsed by the new `ui/ios-parse.ts` into the same `Element` model as Android);
|
|
704
|
+
interaction, keys, and screen size use `idb ui …`; simulator
|
|
705
|
+
screenshots/launch/stop/logs stay on `xcrun simctl` (no idb needed for those).
|
|
706
|
+
Install idb with `brew install idb-companion` + `pip install fb-idb`;
|
|
707
|
+
`vk doctor --ios` verifies it. The old `SimctlDriver` is replaced by `IdbDriver`.
|
|
708
|
+
Documented caveats: `clear` is unsupported (iOS has no per-app data reset),
|
|
709
|
+
`current` returns `(unknown)`, `swipe` duration is not honored, and `log`
|
|
710
|
+
capture is simulator-only.
|
|
711
|
+
- **iOS `vk ai` example** (`example/example-test-ios.md`) — a plain-English Settings
|
|
712
|
+
navigate-and-search smoke test, the iOS counterpart to the Android camera example.
|
|
713
|
+
|
|
714
|
+
## [0.4.1] - 2026-07-07
|
|
715
|
+
|
|
716
|
+
### Added
|
|
717
|
+
- **Published to npm** — install the CLI with `npm install -g verikun` (ships the
|
|
718
|
+
compiled `dist/`, so no build-from-source needed). Added a root MIT `LICENSE`
|
|
719
|
+
file and npm package metadata (`repository`, `homepage`, `bugs`, `keywords`,
|
|
720
|
+
`author`, `publishConfig`).
|
|
721
|
+
|
|
722
|
+
### Changed
|
|
723
|
+
- **Build runs on the `prepare` lifecycle hook** (was `prepublishOnly`), so
|
|
724
|
+
`npm publish`, `npm pack`, and install-from-git all compile `dist/` first —
|
|
725
|
+
fixing an `npm pack` that previously produced a `dist`-less tarball.
|
|
726
|
+
- **`scripts/gen-version.mjs` now also stamps `.claude-plugin/plugin.json`'s
|
|
727
|
+
`version`** from `package.json` (previously a manual sync), guarded by a new
|
|
728
|
+
`tests/plugin-version.test.ts` drift check.
|
|
729
|
+
|
|
730
|
+
## [0.4.0] - 2026-07-06
|
|
731
|
+
|
|
732
|
+
### Added
|
|
733
|
+
- **`clear` command and `launch --clear`** to wipe an app's locally stored data —
|
|
734
|
+
login/session, preferences, caches — resetting it to a just-installed state
|
|
735
|
+
before a flow. Android via `pm clear` (which also force-stops the app);
|
|
736
|
+
recorded as a test-run step like other lifecycle actions. iOS is not supported
|
|
737
|
+
yet (`simctl` has no per-app data reset).
|
|
738
|
+
- **Unit test suite** (`npm test`) for the platform-agnostic core — selector
|
|
739
|
+
matching/auto-healing, the `uiautomator` XML parser, element formatting, the
|
|
740
|
+
PNG downscaler, JUnit/HTML report rendering, argument/duration parsing, and
|
|
741
|
+
the device-shell escaper. It runs on Node's built-in test runner
|
|
742
|
+
(`node:test`), so no test framework is added and the zero-runtime-dependency
|
|
743
|
+
rule holds (dev deps remain `typescript` + `@types/node`). Tests live in
|
|
744
|
+
`tests/*.test.ts`, compile via `tsconfig.test.json` into a gitignored
|
|
745
|
+
`.test-build/`, and need no connected device.
|
|
746
|
+
- **`vk ai <file>`** — run a natural-language test: compiled once to a deterministic
|
|
747
|
+
plan IR, replayed model-free, with the model woken only to repair a drifted selector
|
|
748
|
+
(`src/agent/`). Every run is bounded by default — spend by `--max-cost-usd` (default
|
|
749
|
+
$3) and wall-clock by `--timeout` (default 15m, e.g. `--timeout 5m`) — so a runaway
|
|
750
|
+
compile/repair loop can't spend or hang without limit.
|
|
751
|
+
- **`launch`/`open` `--no-restart`** to bring an app forward without restarting it.
|
|
752
|
+
- **`example/`** — a natural-language `vk ai` example test and README.
|
|
753
|
+
|
|
754
|
+
### Changed
|
|
755
|
+
- `launch`/`open` **restarts by default** (force-stops the app first) so a rerun starts
|
|
756
|
+
from a fresh state instead of a stale screen; `--no-restart` opts out.
|
|
757
|
+
- The `vk ai` plan cache is gated by a compiler fingerprint (verikun version + grammar)
|
|
758
|
+
and written at compile time, so an unchanged test never recompiles and a verikun
|
|
759
|
+
update never replays a plan an older build produced.
|
|
760
|
+
|
|
761
|
+
### Fixed
|
|
762
|
+
- `launch` resolves the launcher activity and uses `am start` instead of `monkey -c
|
|
763
|
+
LAUNCHER` (which hangs on MIUI/HyperOS), and now detects a failed start (stderr + exit
|
|
764
|
+
code), not just stdout.
|
|
765
|
+
- `vk ai` repair can **give up** (terminal failure) instead of substituting a wrong
|
|
766
|
+
element onto a drifted screen and passing falsely.
|
|
767
|
+
- `vk log` hardened: `--since` is validated against the logcat timestamp charset (no
|
|
768
|
+
device-shell injection) and `--out` is confined to the working directory.
|
|
769
|
+
- The `vk ai` trust boundary is tightened to every model-reachable sink, not just `vk
|
|
770
|
+
log`: `screenshot --out` is confined to the working directory (was unguarded), and the
|
|
771
|
+
`launch`/`stop`/`clear` package id is validated against a safe charset so model output
|
|
772
|
+
can't inject into the device shell.
|
|
773
|
+
- `vk ai` rejects an empty compiled/cached plan instead of reporting a green pass for a
|
|
774
|
+
test that ran no steps.
|
|
775
|
+
- `launch --no-restart` is parsed as a boolean, so it works before the package argument
|
|
776
|
+
(`vk launch --no-restart <pkg>`) instead of swallowing it.
|
|
777
|
+
- A model-healed step no longer shows the failed attempt's screenshot/hierarchy in the
|
|
778
|
+
report (it is a pass, not a failure).
|
|
779
|
+
|
|
780
|
+
## [0.3.0] - 2026-06-07
|
|
781
|
+
|
|
782
|
+
### Added
|
|
783
|
+
- **Screenshots are downscaled by default** to a 700px longest edge, so an agent
|
|
784
|
+
reading one back spends far fewer tokens (image cost scales with pixel area)
|
|
785
|
+
while UI text stays legible — a typical capture shrinks ~12× in area. New
|
|
786
|
+
`vk screenshot` flags: `--more` (bump to a 1400px cap when 700 reads too
|
|
787
|
+
coarse), `--max <px>` (an exact cap), and `--full` (write the original).
|
|
788
|
+
`VERIKUN_SHOT_MAX_EDGE` changes the default globally.
|
|
789
|
+
- A dependency-free, pure-Node PNG resampler (`src/image.ts`, built on
|
|
790
|
+
`node:zlib`). PNGs it can't safely resample (palette, 16-bit, interlaced) are
|
|
791
|
+
written through untouched, so a capture is never corrupted — only left
|
|
792
|
+
full-size. It never upscales.
|
|
793
|
+
|
|
794
|
+
### Fixed
|
|
795
|
+
- `vk --version` and `package-lock.json` now report the actual package version
|
|
796
|
+
(both had drifted behind `package.json`).
|
|
797
|
+
|
|
798
|
+
## [0.2.0] - 2026-06-05
|
|
799
|
+
|
|
800
|
+
### Added
|
|
801
|
+
- **Test runs**: recordable actions form a run that archives to a JUnit
|
|
802
|
+
`report.xml` plus a self-contained HTML report
|
|
803
|
+
(`vk run start|status|archive|clear`), capturing timings, the resolved
|
|
804
|
+
identifier per step, and — on failure — a screenshot and the page's UI
|
|
805
|
+
hierarchy. `vk run archive` exits non-zero on failures, so it doubles as a CI
|
|
806
|
+
gate. An implicit run auto-starts on the first action and rolls over (archiving
|
|
807
|
+
the old one) on a device or session change, or after idle.
|
|
808
|
+
- **Selector auto-wait**: `tap`, `text`, `find`, `assert`, and `swipe --on` retry
|
|
809
|
+
the lookup for up to 5s instead of failing on the first miss. `--wait <dur>`
|
|
810
|
+
tunes the window (`8s`, `800ms`, bare ms), `--no-wait` / `--wait 0` fails fast,
|
|
811
|
+
and `assert --gone` waits for disappearance. Ambiguous matches are never
|
|
812
|
+
waited on.
|
|
813
|
+
|
|
814
|
+
## [0.1.0] - 2026-06-03
|
|
815
|
+
|
|
816
|
+
### Added
|
|
817
|
+
- Initial release. Drive a connected Android device (and partial iOS via
|
|
818
|
+
`simctl`) Puppeteer-style: `tap`, `text`, `type`, `swipe`, `key`,
|
|
819
|
+
`screenshot`, `launch`, `stop`, plus semantic hierarchy inspection (`ui`,
|
|
820
|
+
`find`, `assert`, `wait`) with auto-healing selectors (`@id` / `text:` /
|
|
821
|
+
`desc:` / `class:`), a machine-readable exit-code contract, and `--json`
|
|
822
|
+
output everywhere.
|