verikun 0.28.0 → 0.29.0

package/.claude/skills/verikun/SKILL.md

@@ -514,6 +514,11 @@ installs again, warning on stderr that **that build's app data is gone**. A same
 install still keeps its data. Do not treat the warning as a failure — it is how the
 install succeeded. iOS has no equivalent recovery; the install just fails there.
 
+**`vk install` also allows a version downgrade by default, when the build is debuggable.**
+A build with a lower version code than what's already on the device installs normally
+instead of failing with `INSTALL_FAILED_VERSION_DOWNGRADE`. A release-signed build is a
+genuine exception — Android still refuses it, and that is not something to retry.
+
 `vk server --devices all` (or `all-android` / `all-ios` / a serial list) serves a **pool**
 from one address. Each run leases one device for its whole life, so a run's steps and
 repairs always land on the same phone. `vk install --server` then installs on every device;

package/CHANGELOG.md

@@ -6,6 +6,25 @@ All notable changes to this project are documented here. The format is based on
 
 ## [Unreleased]
 
+## [0.29.0] - 2026-09-16
+
+### Changed
+- **`vk install`** (Android) allows a version downgrade by default (`adb install -d`),
+  instead of failing with `INSTALL_FAILED_VERSION_DOWNGRADE`, when the build is debuggable.
+
+## [0.28.1] - 2026-09-15
+
+### Fixed
+- **`vk install --server`** sheds a pooled device whose install stalls, while the client now
+  honors its 15-minute request budget. ([#143])
+
+### Changed
+- **Docs rule**: a bug fix is changelog-only; `docs/` changes only when a contract moves or a page
+  went false. ([#140])
+
+[#140]: https://github.com/ddikman/verikun/issues/140
+[#143]: https://github.com/ddikman/verikun/issues/143
+
 ## [0.28.0] - 2026-09-14
 
 A device that has gone now leaves a pooled `vk server` instead of being dealt out until
@@ -26,7 +45,6 @@ somebody notices.
   partial failure. Older clients see a success with an unknown field. ([#139])
 
 [#139]: https://github.com/ddikman/verikun/issues/139
-
 ## [0.27.1] - 2026-09-14
 
 A hierarchy read the device killed is now waited out instead of ending the test.

package/dist/agent/remote.js

@@ -17,18 +17,21 @@ exports.remoteDeviceList = remoteDeviceList;
 exports.createRemoteBackend = createRemoteBackend;
 const node_fs_1 = require("node:fs");
 const node_crypto_1 = require("node:crypto");
+const node_http_1 = require("node:http");
+const node_https_1 = require("node:https");
 const node_path_1 = require("node:path");
 const errors_1 = require("../errors");
+const install_timeouts_1 = require("../install-timeouts");
 const output_1 = require("../output");
 const rpc_1 = require("../rpc");
 /**
- * The ceiling NONE of the per-call timeouts below can exceed, whatever they say.
+ * The ceiling fetch-based calls below cannot exceed, whatever their own budgets say.
  *
  * Node's global `fetch` is undici, whose `headersTimeout` and `bodyTimeout` both default to
  * 300s, and there is no dependency-free way to raise them: a `dispatcher` needs `undici`
- * itself, which is bundled but not importable. The `AbortController` below is therefore a
- * FLOOR on how long a call may take, never a ceiling — `EXEC_TIMEOUT_MS` says 600s and gets
- * 300s.
+ * itself, which is bundled but not importable. Long-running install uploads therefore use
+ * Node's built-in http/https client instead; its explicit 15-minute timer is the real
+ * upload-and-response ceiling. The other AbortControllers remain a floor when set above 300s.
  *
  * MEASURED on Node v20.20.2 against a server that held its headers for 310s: the fetch
  * rejected at 301s with `TypeError: fetch failed`, cause `HeadersTimeoutError`, code
@@ -39,11 +42,10 @@ const rpc_1 = require("../rpc");
 const FETCH_HEADERS_CEILING_MS = 300_000;
 // Per-call ceilings. exec is generous: a single leaf may legitimately block for its
 // whole auto-wait window or an explicit `wait --timeout`, plus device time. Anything here
-// above FETCH_HEADERS_CEILING_MS is aspirational — see that constant.
+// above FETCH_HEADERS_CEILING_MS is aspirational unless it uses requestWithNodeHttp.
 const HEALTH_TIMEOUT_MS = 10_000;
 const ELEMENTS_TIMEOUT_MS = 60_000;
 const EXEC_TIMEOUT_MS = 10 * 60_000;
-const INSTALL_TIMEOUT_MS = 15 * 60_000;
 const DEVICE_LIST_TIMEOUT_MS = 30_000;
 // Meant to sit above the server's own 4-minute boot ceiling, so the SERVER reports why a
 // boot timed out rather than the client aborting first. It is exactly AT
@@ -52,6 +54,54 @@ const DEVICE_LIST_TIMEOUT_MS = 30_000;
 const DEVICE_START_TIMEOUT_MS = 5 * 60_000;
 const DEVICE_STOP_TIMEOUT_MS = 60_000;
 const trimUrl = (url) => url.replace(/\/+$/, '');
+/**
+ * A dependency-free request path whose caller-owned timeout covers BOTH upload and response.
+ *
+ * Global fetch cannot wait past undici's fixed 300s header/body ceilings. An install may
+ * legitimately need 15 minutes, so it uses Node's native protocol clients instead of claiming
+ * a budget the transport cannot honor. Buffering the response preserves the Response boundary
+ * used below; install responses are small JSON descriptors, never artifacts.
+ */
+function requestWithNodeHttp(url, method, headers, body, timeoutMs) {
+    const endpoint = new URL(url);
+    const send = endpoint.protocol === 'http:' ? node_http_1.request : endpoint.protocol === 'https:' ? node_https_1.request : null;
+    if (!send)
+        return Promise.reject(new Error(`unsupported server protocol '${endpoint.protocol}'`));
+    const payload = body === undefined ? undefined : Buffer.isBuffer(body) ? body : Buffer.from(body);
+    const requestHeaders = {
+        ...headers,
+        ...(payload ? { 'content-length': String(payload.byteLength) } : {}),
+    };
+    return new Promise((resolve, reject) => {
+        let settled = false;
+        let timer;
+        const finish = (fn) => {
+            if (settled)
+                return;
+            settled = true;
+            if (timer)
+                clearTimeout(timer);
+            fn();
+        };
+        const req = send(endpoint, { method, headers: requestHeaders }, (res) => {
+            const chunks = [];
+            res.on('data', (chunk) => chunks.push(Buffer.from(chunk)));
+            res.on('error', (e) => finish(() => reject(e)));
+            res.on('aborted', () => finish(() => reject(new Error('the server aborted its response'))));
+            res.on('end', () => {
+                const status = res.statusCode ?? 500;
+                finish(() => resolve(new Response(Buffer.concat(chunks), { status })));
+            });
+        });
+        req.on('error', (e) => finish(() => reject(e)));
+        timer = setTimeout(() => {
+            const timeout = Object.assign(new Error('This operation was aborted'), { name: 'AbortError' });
+            req.destroy(timeout);
+        }, timeoutMs);
+        timer.unref?.();
+        req.end(payload);
+    });
+}
 /**
  * Turn a non-2xx into the error the caller sees.
  *
@@ -132,24 +182,31 @@ class RemoteTransport {
             h.authorization = `Bearer ${this.opts.authKey}`;
         return h;
     }
-    async request(method, path, body, timeoutMs, extraHeaders = {}) {
+    async request(method, path, body, timeoutMs, extraHeaders = {}, transport = 'fetch') {
         const url = `${this.base}${path}`;
-        const controller = new AbortController();
-        const timer = setTimeout(() => controller.abort(), timeoutMs);
+        let timer;
         let res;
         try {
-            res = await fetch(url, {
-                method,
-                headers: this.headers(extraHeaders),
-                body,
-                signal: controller.signal,
-            });
+            if (transport === 'node-http') {
+                res = await requestWithNodeHttp(url, method, this.headers(extraHeaders), body, timeoutMs);
+            }
+            else {
+                const controller = new AbortController();
+                timer = setTimeout(() => controller.abort(), timeoutMs);
+                res = await fetch(url, {
+                    method,
+                    headers: this.headers(extraHeaders),
+                    body,
+                    signal: controller.signal,
+                });
+            }
         }
         catch (e) {
             throw new errors_1.CliError(`cannot reach verikun server at ${url} (${transportReason(e, timeoutMs)})`, 3);
         }
         finally {
-            clearTimeout(timer);
+            if (timer)
+                clearTimeout(timer);
         }
         if (!res.ok) {
             const body = await readBody(res);
@@ -262,11 +319,11 @@ function createRemoteBackend(opts, health) {
                 throw new errors_1.CliError(`install: cannot read '${appPath}' (${e.message})`, 2);
             }
             const sha256 = (0, node_crypto_1.createHash)('sha256').update(buf).digest('hex');
-            const res = await t.request('POST', '/v1/install', buf, INSTALL_TIMEOUT_MS, {
+            const res = await t.request('POST', '/v1/install', buf, install_timeouts_1.REMOTE_INSTALL_TIMEOUT_MS, {
                 'content-type': 'application/octet-stream',
                 'x-verikun-ext': ext,
                 'x-verikun-sha256': sha256,
-            });
+            }, 'node-http');
             // Install is the one operation the server replays elsewhere, so a move here means
             // the build DID land — on a different device than the one we started with.
             if (res.deviceChanged)

package/dist/device/failover.js

@@ -33,6 +33,7 @@ const ARTIFACT_RULES = [
 const TRANSIENT_RULES = [[/INSTALL_FAILED_ABORTED/, 'the install session was aborted']];
 // --- fast paths: name the reason and skip the probe. NEVER the gate. ----------
 const UNREACHABLE_RULES = [
+    [/stopped responding.*install/i, 'the device stopped responding during install'],
     [/device (?:'[^']*' )?not found/i, 'the device is not attached'],
     [/no devices\/emulators found/i, 'no device is attached'],
     [/device offline/i, 'the device is offline'],
@@ -49,6 +50,9 @@ const DEVICE_STATE_RULES = [
     // `install` already tried removing it and reinstalling (drivers/adb.ts); reaching here
     // means that did not work, so the wording must not imply nothing was attempted.
     [/INSTALL_FAILED_UPDATE_INCOMPATIBLE/, 'a differently-signed build could not be replaced on the device'],
+    // `install` already retries with `-d` (drivers/adb.ts); reaching here means the build
+    // or device isn't debuggable, so Android is still refusing — a genuine per-device
+    // conflict, not a sign the retry never ran.
     [/INSTALL_FAILED_VERSION_DOWNGRADE/, 'the device holds a newer build of this package'],
     [/INSTALL_FAILED_ALREADY_EXISTS/, 'the package is already installed on the device'],
     [/INSTALL_FAILED_DUPLICATE_PERMISSION/, 'another app on the device declares one of these permissions'],
@@ -166,8 +170,9 @@ function classifyInstallFailure(e) {
         if (named)
             return { move: true, kind: 'device-state', reason: named };
         // The inversion. Not in the denylist ⇒ it is about the device, even though we have
-        // never seen this wording. Bounded by MAX_FAILOVER_HOPS, and the caller reports the
-        // FIRST device's error on exhaustion, so a wrong guess costs time, not diagnosis.
+        // never seen this wording. Bounded by the request-derived install hop limit, and the
+        // caller reports the FIRST device's error on exhaustion, so a wrong guess costs time,
+        // not diagnosis.
         return { move: true, kind: 'device-state', reason: 'the device could not install this build', unclassified: true };
     }
     return classify(e, UNKNOWN_STAY);

package/dist/drivers/adb.js

@@ -1104,21 +1104,27 @@ class AdbDriver {
             throw new errors_1.CliError(signatureConflictHelp(appPath, pkg, 'retry-failed', second), 3);
     }
     /**
-     * One `adb install -r` attempt: null on success, else adb's collapsed output.
+     * One `adb install -r -d` attempt: null on success, else adb's collapsed output.
      *
      * Returns rather than throws because the caller has to READ the failure to decide
      * whether it is recoverable — and a thrown-and-caught CliError would put the final
      * message's own prefix inside the message it is composing.
      *
      * `-r` reinstalls over an existing package keeping its data (the common
-     * update-the-build-under-test case). A large APK can legitimately take minutes to
-     * stream + install, so the timeout is far above the 30s default. adb reports failures
-     * both as a non-zero exit AND as a `Failure [REASON]` line on stdout with exit 0
-     * (varies by adb version) — check both, and require `Success` positively rather than
-     * merely inferring it from the absence of `Failure`.
+     * update-the-build-under-test case). `-d` allows a lower versionCode to replace a
+     * higher one already on the device — unconditional here, not a retry like the
+     * signature-conflict replace below, because it carries no data-loss trade-off (inert
+     * when there is no downgrade, and keeps data the same way `-r` does when there is).
+     * Android only honors it for a debuggable build, so a release-signed downgrade still
+     * surfaces `INSTALL_FAILED_VERSION_DOWNGRADE` unresolved — see the comment on that
+     * code in `device/failover.ts`. A large APK can legitimately take minutes to stream +
+     * install, so the timeout is far above the 30s default. adb reports failures both as a
+     * non-zero exit AND as a `Failure [REASON]` line on stdout with exit 0 (varies by adb
+     * version) — check both, and require `Success` positively rather than merely inferring
+     * it from the absence of `Failure`.
      */
     tryInstall(appPath) {
-        const r = (0, exec_1.runText)(ADB, this.withSerial(['install', '-r', appPath]), { timeout: 10 * 60 * 1000 });
+        const r = (0, exec_1.runText)(ADB, this.withSerial(['install', '-r', '-d', appPath]), { timeout: 10 * 60 * 1000 });
         const combined = `${r.stdout}\n${r.stderr}`;
         if (r.code !== 0 || /^Failure\b/im.test(combined) || !/^Success\b/im.test(combined)) {
             return combined.replace(/\s+/g, ' ').trim() || `exit code ${r.code}`;

package/dist/install-timeouts.js

@@ -0,0 +1,12 @@
+"use strict";
+// The remote-install time budget is shared by the client that owns the request and the
+// server that may retry it. Keeping the numbers together prevents a server-side retry plan
+// that can never finish before the client gives up.
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.MAX_INSTALL_FAILOVER_HOPS = exports.INSTALL_DEVICE_TIMEOUT_MS = exports.REMOTE_INSTALL_TIMEOUT_MS = void 0;
+/** Whole upload + install request, enforced by the remote client. */
+exports.REMOTE_INSTALL_TIMEOUT_MS = 15 * 60_000;
+/** One device gets four minutes; a measured ~170s large install still has useful margin. */
+exports.INSTALL_DEVICE_TIMEOUT_MS = 4 * 60_000;
+/** Three attempts consume at most 12m, leaving 3m of the request budget for the upload. */
+exports.MAX_INSTALL_FAILOVER_HOPS = Math.max(0, Math.floor(exports.REMOTE_INSTALL_TIMEOUT_MS / exports.INSTALL_DEVICE_TIMEOUT_MS) - 1);

package/dist/server.js

@@ -45,6 +45,7 @@ const node_stream_1 = require("node:stream");
 const promises_1 = require("node:stream/promises");
 const args_1 = require("./args");
 const errors_1 = require("./errors");
+const install_timeouts_1 = require("./install-timeouts");
 const drivers_1 = require("./drivers");
 const manager_1 = require("./companion/manager");
 const claims_1 = require("./device/claims");
@@ -86,10 +87,6 @@ const INSTALL_BODY_CAP = 512 * 1024 * 1024; // 512 MB app build
 // to survive a client-side compile/repair pause, short enough that a crashed
 // caller doesn't wedge the device.
 const LOCK_IDLE_MS = 5 * 60 * 1000;
-// How many times ONE request may move device. 2 moves = 3 devices tried, which sits
-// comfortably inside the client's 15-minute install ceiling at ~1 minute an install,
-// while a farm of ten wedged emulators cannot burn ten installs inside one request.
-const MAX_FAILOVER_HOPS = 2;
 // How often to ask whether the host's adb server has rotted. Generous on purpose: the
 // check shells out to `log show` (~1s) and the condition it looks for accumulates over
 // DAYS, so a tight interval would buy nothing and spend host time on every idle server.
@@ -108,6 +105,16 @@ const ADB_RECYCLE_CHECK_MS = 10 * 60 * 1000;
  */
 const RECONCILE_INTERVAL_MS = 60_000;
 const PROBE_RETRY_MS = 1000;
+/** Kept distinct so an all-device timeout is never mistaken for evidence of a bad build. */
+class InstallAttemptTimeoutError extends errors_1.CliError {
+    constructor(serial, timeoutMs) {
+        const duration = timeoutMs >= 1000 ? `${Math.round(timeoutMs / 1000)}s` : `${timeoutMs}ms`;
+        super(`device ${serial} stopped responding: install exceeded its ${duration} per-device deadline`, 3);
+        this.name = 'InstallAttemptTimeoutError';
+    }
+}
+/** Failover wraps an exhausted attempt in HttpError, so retain a wire-safe marker too. */
+const isInstallAttemptTimeout = (e) => e instanceof InstallAttemptTimeoutError || (e instanceof Error && /per-device deadline/.test(e.message));
 // Deliberately below the client's 5-minute ceiling, so a slow boot is reported by the
 // side that knows WHY ("did not finish booting within 240s") rather than as a generic
 // client-side abort.
@@ -1251,7 +1258,7 @@ function buildServer(config) {
      * On exhaustion it throws the FIRST device's error, never the last. That inversion is
      * what makes move-by-default safe: a wrong move costs time, not the diagnosis.
      */
-    async function installWithFailover(serial, tmpPath) {
+    async function installWithFailover(serial, tmpPath, timedOut) {
         let change;
         let moves = 0;
         let firstError;
@@ -1291,7 +1298,7 @@ function buildServer(config) {
                 // are still on our disk), so it hops rather than throwing a bare 503 that never
                 // reaches the failover machinery at all.
                 const gone = firstError ?? new server_http_1.HttpError(503, `device ${from} is no longer attached`, 3);
-                if (!config.failover || hop >= MAX_FAILOVER_HOPS)
+                if (!config.failover || hop >= install_timeouts_1.MAX_INSTALL_FAILOVER_HOPS)
                     throw gone;
                 // `unreachable` is the literal truth — it is not in the pool — and it is also
                 // inert here: `shrink` sees a non-member and takes its cleanup tail either way.
@@ -1299,7 +1306,28 @@ function buildServer(config) {
                 continue;
             }
             try {
-                await handle.install(tmpPath);
+                const timeoutMs = config.installAttemptMs ?? install_timeouts_1.INSTALL_DEVICE_TIMEOUT_MS;
+                await new Promise((resolve, reject) => {
+                    let settled = false;
+                    const finish = (fn) => {
+                        if (settled)
+                            return;
+                        settled = true;
+                        clearTimeout(timer);
+                        fn();
+                    };
+                    const timer = setTimeout(() => {
+                        const timeout = new InstallAttemptTimeoutError(from, timeoutMs);
+                        // A race alone would leave the worker blocked in adb forever. Retiring it is
+                        // what releases the transport and makes the deadline a recovery mechanism.
+                        timedOut.add(from);
+                        quarantineDevice(from, timeout.message);
+                        pool.retire(from);
+                        finish(() => reject(timeout));
+                    }, timeoutMs);
+                    timer.unref?.();
+                    void handle.install(tmpPath).then(() => finish(resolve), (e) => finish(() => reject(e)));
+                });
                 return { change, moves };
             }
             catch (e) {
@@ -1310,8 +1338,18 @@ function buildServer(config) {
                 noteVerdict(verdict, e, 'install');
                 // The artifact is broken / the caller is wrong / failover is off / we are out of
                 // hops: report the first failure unchanged, exactly as before this feature.
-                if (!verdict.move || !config.failover || hop >= MAX_FAILOVER_HOPS)
+                if (!verdict.move || !config.failover || hop >= install_timeouts_1.MAX_INSTALL_FAILOVER_HOPS) {
+                    // A timeout retired its worker before this catch. If it cannot move, finish the
+                    // cleanup that pickFailoverDevice would otherwise own; especially, never leave
+                    // a host-global claim pinned to a worker that no longer exists.
+                    if (isInstallAttemptTimeout(e)) {
+                        evictHoldersOf(from, `${from} left the pool after its install timed out`);
+                        (0, manager_1.releaseCompanionOn)(from);
+                        if ((0, claims_1.claimsEnabled)(claimEnv))
+                            (0, claims_1.releaseClaim)(from, { ...claimOpts, mineOnly: true });
+                    }
                     throw firstError;
+                }
                 from = await hopOrThrow(verdict.reason, firstError, verdict.kind);
             }
         }
@@ -1361,9 +1399,10 @@ function buildServer(config) {
                 throw new server_http_1.HttpError(503, `no device is left to install onto${lostDevice ? ` — last loss: ${lostDevice}` : ''}`, 3);
             }
             (0, output_1.err)(`[server] install: received ${size} bytes (.${ext}), installing on ${targets.join(', ')}…`);
+            const timedOut = new Set();
             const outcomes = await Promise.all(targets.map(async (serial) => {
                 try {
-                    return { serial, ...(await installWithFailover(serial, tmpPath)), error: null };
+                    return { serial, ...(await installWithFailover(serial, tmpPath, timedOut)), error: null };
                 }
                 catch (e) {
                     return { serial, change: undefined, moves: 0, error: e };
@@ -1387,9 +1426,21 @@ function buildServer(config) {
                     // polarity, since the device side is open-ended and OEM-specific. That is right
                     // for ONE device failing; applied to every device at once it condemns the whole
                     // pool for what this very branch has just concluded is a bad build. Undo them.
-                    const condemned = targets.filter((t) => quarantine.delete(t));
-                    if (condemned.length) {
-                        (0, output_1.err)(`[server] install: failed on every device, so the build is the suspect — un-quarantining ${condemned.join(', ')}`);
+                    const hasTimeout = timedOut.size > 0;
+                    if (!hasTimeout) {
+                        // No lane succeeded and none hung: the common input is now the stronger
+                        // suspect, so undo the per-device quarantines made by the move-by-default
+                        // classifier.
+                        const condemned = targets.filter((t) => quarantine.delete(t));
+                        if (condemned.length) {
+                            (0, output_1.err)(`[server] install: failed on every device, so the build is the suspect — un-quarantining ${condemned.join(', ')}`);
+                        }
+                    }
+                    else {
+                        // A deadline is direct evidence about a device, not the artifact. Preserve
+                        // every quarantine when any lane timed out; otherwise the next request could
+                        // immediately be dealt the same wedged worker again.
+                        (0, output_1.err)('[server] install: every device failed and at least one timed out — keeping the device quarantines');
                     }
                     throw failed[0].error;
                 }
@@ -1412,11 +1463,15 @@ function buildServer(config) {
                 skipped = failed.map((f) => {
                     const serial = landedOn(f);
                     const reason = (0, server_http_1.firstLine)(f.error.message);
-                    if (pool.serials().includes(serial)) {
+                    const serving = pool.serials().includes(serial);
+                    if (serving)
                         pool.retire(serial);
-                        // Same two rules as the shed in `shrink`: `degraded` is for MEMBERS, and a
-                        // device that is not serving belongs in `quarantine` — which `rejoinDevice`
-                        // clears on the evidence of a worker that started and a build that installed.
+                    // Same two rules as the shed in `shrink`: `degraded` is for MEMBERS, and a
+                    // device that is not serving belongs in `quarantine` — which `rejoinDevice`
+                    // clears on the evidence of a worker that started and a build that installed.
+                    // The timeout path retired its worker already, but still needs this common
+                    // bookkeeping tail. Other already-absent failures keep their existing handling.
+                    if (serving || isInstallAttemptTimeout(f.error)) {
                         degraded.delete(serial);
                         quarantineDevice(serial, `did not take the current build — ${reason}`);
                         evictHoldersOf(serial, `${serial} left the pool without the current build`);

package/dist/version.js

@@ -3,4 +3,4 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.VERSION = void 0;
 // GENERATED by scripts/gen-version.mjs from package.json's "version" at build time
 // (the `prebuild` script). Do NOT edit by hand; bump package.json instead.
-exports.VERSION = '0.28.0';
+exports.VERSION = '0.29.0';

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "verikun",
-  "version": "0.28.0",
+  "version": "0.29.0",
   "description": "Drive Android emulators/devices and iOS simulators for AI agents: tap, type, swipe, screenshot, and inspect the UI hierarchy by semantic identifiers — like Puppeteer for native apps.",
   "keywords": [
     "android",