simframe 0.12.2 → 0.13.0

package/README.md

@@ -30,6 +30,41 @@ flows run in one call, screens the agent has seen before are answered from
 memory, and every answer is text with tap points in it. Nothing returns an
 image unless you ask for one.
 
+### It is also a study case, and the numbers are the point
+
+Almost everything here was decided by a measurement rather than by an argument,
+and several of those measurements **reversed a decision that had already
+shipped**. They are written down in full, with what we expected beforehand, what
+it cost to find out, and the mistakes made getting there — because a benchmark
+that only records the winner teaches nothing.
+
+| | |
+| --- | --- |
+| [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md) | the record: N, median, p95, machine, every round |
+| [`docs/EXPERIMENTS.md`](docs/EXPERIMENTS.md) | what we believed *before* measuring, and the fourteen times it was wrong |
+| [`docs/DEFERRED.md`](docs/DEFERRED.md) | every known defect, open or closed, with the evidence |
+| [`docs/ARTICLE.md`](docs/ARTICLE.md) | the argument the whole thing adds up to |
+
+The headline results, all free to reuse:
+
+- **A warm transition graph removes ~27% of an agent's model round trips** —
+  isolated with a three-pass field experiment, and corroborated to three points
+  by an independent estimate. Operator knowledge is worth twice that, and no
+  tool here carries it yet. ([§14](docs/EXPERIMENTS.md))
+- **A bigger local judge is not a better one.** `qwen3:14b` scored *lower* than
+  `qwen3:8b` — 82% against 91% — while being 79% larger and 62% slower. A
+  one-line threshold on a number already computed beat all of them at **95% and
+  zero latency**.
+- **Every interactive element came from the accessibility tree — 72 of 72 — and
+  83% of them have no text at all.** Screen *recognition*, though, did not need
+  the tree once. The ladder splits there: semantics need a tree, identity needs
+  pixels.
+- **Capture is ~31× faster than a screenshot** for the same work, and the
+  capture primitive itself about a thousand times faster.
+- **Three independent reporters converged on one thing**: a refusal is cheap and
+  a confident wrong answer is expensive. Every serious bug they found was a
+  component reporting more certainty than it had.
+
 ## What changed, measured
 
 Same four-tab navigation flow, on a real production app:
@@ -177,13 +212,30 @@ was otherwise impossible.
 
 **What is measured and what is not.** The ranker: 5 of 6 top-1 on hand-written
 cases, median 564 ms warm, and on a real exploration it went to the right region
-in two steps where reading order wandered into version strings. The supervisor:
-correct on four real batch-killers once the plan briefed it, 689–751 ms warm —
-**on a bench, not in the field.** `ax-first` made no measurable difference to how
-an agent drove a real app, with one small regression and one small win. Numbers
-and conditions are in [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md); the judgements,
-including a phase cancelled by its own measurement, are in
-[`docs/DECISIONS.md`](docs/DECISIONS.md).
+in two steps where reading order wandered into version strings. `ax-first` made
+no measurable difference to how an agent drove a real app, with one small
+regression and one small win.
+
+**The supervisor, on 22 labelled failures from a seeded React Native app** —
+and the number to judge it by is round trips, not accuracy:
+
+| | |
+|---|---|
+| handled locally, no round trip | **9 of 22** |
+| escalated to Claude | 13 of 22 |
+| a model round trip, measured in the field | 10–16 s |
+
+So roughly 90–145 s saved on that population. Accuracy was 77 / 82 / 86% across
+three runs of the *same* questions — it is **not deterministic**, and every
+earlier single-run figure in this project carried that spread without reporting
+it. All of its errors were `wait` where `stop` was right, which is the cheap
+direction: a wrong `wait` costs a settle and a re-run, and `stop` already means
+"hand back to Claude with the unattempted steps" rather than "give up".
+
+**Still a bench, not the field.** These are fixtures we designed, on a testbed we
+built, labelled by the person who then scored them. Numbers and conditions are in
+[`docs/BENCHMARKS.md`](docs/BENCHMARKS.md); the judgements, including a phase
+cancelled by its own measurement, are in [`docs/DECISIONS.md`](docs/DECISIONS.md).
 
 **And the measurements that changed our minds** are in
 [`docs/EXPERIMENTS.md`](docs/EXPERIMENTS.md), with what we expected beforehand
@@ -212,6 +264,15 @@ The Ollama arm is an **experiment, not a recommendation**: off unless named,
 no weights shipped, no dependency added, and every arm reads the same briefing
 out of `native/supervise.swift` so no arm is answering a different question.
 
+**Two further results, both negative, both worth more than the table.** A
+*cascade* — free rule first, model where it is unsure, Claude after that — was
+worse at every abstention band we tried (95% for the rule alone; 91%, 86%, 82%
+as more was handed to the model). And giving the model a fourth word, `abstain`,
+cost the Apple arm about a third of its accuracy (77/82/86% → 45/50/55%) **while
+it never used the word once**. One added paragraph, nothing else changed. So the
+fourth word ships off, and "add an abstain token" became "find a judge that will
+use one".
+
 The supervisor's whole vocabulary is three words on purpose. It cannot invent a
 step, skip one, substitute a target or continue past an unexpected screen — not
 because a threshold forbids it but because those are not answers it can give.
@@ -538,6 +599,29 @@ simframe flow save checkout ./checkout.json
 simframe flow run checkout
 ```
 
+### What the memory is worth, isolated
+
+Three agent sessions drove the same task family on the same production app. The
+third existed only to hold a variable still: pass 2 beat pass 1 by twenty-one
+model round trips, but it had a warm graph *and* an operator who had already
+driven the app once, so the gap could belong to either.
+
+| pass | operator | graph | round trips |
+| --- | --- | --- | --- |
+| 1 | fresh | cold | **33** |
+| 3 | fresh | **warm** | **24** |
+| 2 | experienced | warm | **12** |
+
+**The graph is worth ~27%** of the round trips. Operator knowledge is worth the
+rest — half of everything left after the graph had taken its share. A second
+reporter, unable to isolate the graph, derived **~23%** from escalation rate per
+step without knowing that number.
+
+One app, one operator per pass, and round trips are not seconds. What it settles
+is the direction: the memory is real, it is the smaller half, and the larger half
+is what an operator learns and no tool here carries between sessions yet. Full
+working in [`docs/EXPERIMENTS.md`](docs/EXPERIMENTS.md) §14.
+
 ## Does this work on *your* app?
 
 Nothing in simframe is written for a particular app. What varies between apps is
@@ -553,9 +637,27 @@ Run `simframe ui` on any screen to see exactly what simframe can see, with each
 target marked `ax` or `ocr`. If something you can read is not listed, that is a
 bug worth reporting.
 
+A control the app **declares but never names** — an icon-only overflow menu, a
+back chevron — is listed with its coordinates and tappable by `#ref` rather than
+by name, and the map counts how many such controls are on the screen. A view the
+app never declared accessible at all is invisible to any accessibility tree, ours
+included, and the same line says so: a count of zero on a screen that has one
+would be the more expensive answer.
+
+**What a settle can and cannot see.** Stillness is decided from a mean over a
+grid of the frame, and a small animation does not move a mean. Measured on a
+screen built to never settle — a spinner, frames every 77–95 ms — the mean
+difference was **0.00196** against a 0.004 threshold while one cell moved by
+**0.0275**, and `stableForMs` reported **79 seconds** of stillness. So a settle
+can return satisfied while part of the screen is still moving, and it now says
+so: `settled after 63ms (a 13x13 region is still animating)`. A settle that
+times out names where the movement is instead. Refusing to settle on any
+animation would be right for a spinner and wrong for a blinking cursor, so the
+disagreement is reported rather than resolved by a guessed threshold.
+
 Two honest caveats. OCR reads **text**, so a purely graphical icon with no label
-is invisible to both paths — use `sim_ui` to get its coordinates from the tree,
-or tap by position. And the confirm-button vocabulary (`APPLY`, `OK`, `SAVE`,
+is invisible to both paths — the tree still gives you its coordinates, and
+`#ref` still taps it. And the confirm-button vocabulary (`APPLY`, `OK`, `SAVE`,
 `DONE`…) is English; a localised UI needs those words extended.
 
 ## Measured
@@ -650,6 +752,18 @@ accepted as the name that loop used to have.
   a support banner was swallowing — a banner that was *in the element list the
   same call printed*. Nothing said "there is something at y≈753 and you started
   at y=750".
+- **A control half off the edge is offered, clamped, and says so.** A filter
+  chip with 29 pt of itself visible used to be dropped — the off-screen test
+  reads the *centre* — and OCR's reading of that same sliver, a box labelled
+  `Flc`, was printed in its place. So the map did not merely omit a control; it
+  offered a meaningless name for it at an ordinary-looking coordinate. Now:
+
+  ```
+  #6 button 387,191  Flowering  (partly off-screen — the coordinate is the middle of the visible part)
+  ```
+
+  `scrollTo` is deliberately unchanged: a clipped element is still "not in
+  view", so it still scrolls rather than calling a sliver good enough.
 
 ## CLI
 

package/native/supervise.swift

@@ -40,7 +40,40 @@ struct Judgement {
     var decision: Decision
 }
 
+/**
+ * The same question with a fourth answer: "I cannot tell".
+ *
+ * A second type rather than a fourth case on `Decision`, because the two have
+ * to be *comparable*. `@Generable` fixes the answer space at compile time, so a
+ * binary that can only be asked one way cannot measure whether the token helps
+ * or merely moves the errors around — and "does the model abstain on the ones
+ * it gets wrong, or at random" is the whole question. One binary, both
+ * vocabularies, chosen per request.
+ *
+ * `abstain` is not a new capability and cannot become one. It means "behave as
+ * if there is no supervisor", which is exactly the `null` the caller already
+ * handles on every failure path. The answer space stays the safety property:
+ * this strictly shrinks what the model can cause to happen.
+ */
+@available(macOS 26.0, *)
+@Generable
+enum CautiousDecision: String {
+    case wait
+    case retry
+    case stop
+    case abstain
+}
+
+@available(macOS 26.0, *)
+@Generable
+struct CautiousJudgement {
+    @Guide(description: "wait if the screen is still arriving, retry if the same step should be attempted again, stop if nothing further can work, abstain if you cannot tell from what you were given")
+    var decision: CautiousDecision
+}
+
 struct Situation: Decodable {
+    /// Ask with the fourth word available. Absent means the shipped three.
+    let mayAbstain: Bool?
     let goal: String?
     let step: String
     let expected: String?
@@ -121,6 +154,19 @@ func serve() async {
     are not being asked what to do, only whether this can proceed. Answer with \
     the decision alone.
     """
+
+    // The same brief with the fourth word, built from the first so the two
+    // cannot drift. Everything above applies; this adds one rule and changes
+    // nothing else, which is what makes the comparison mean something.
+    let abstainInstructions = instructions + "\n\n" + """
+    There is a fourth answer: abstain. Use it when the evidence you were \
+    given does not distinguish these cases — when you would be guessing. \
+    Abstaining is not a failure and it is not penalised: it hands the decision \
+    to someone with more context, which costs one step. A wrong stop abandons a \
+    plan that would have worked, and a wrong wait spends a timeout on a screen \
+    that will never change. Prefer abstain to either. Do not abstain merely \
+    because the situation is unusual; abstain because the evidence is absent.
+    """
     // Warm up front so the first real judgement does not pay model load —
     // measured at ~880ms against ~600ms warm.
     //
@@ -133,7 +179,17 @@ func serve() async {
     // The window, reported rather than assumed. 4,096 has been a documented
     // constant we repeated; since 26.4 it is queryable, so it is now read from
     // the model and handed to the caller, who prints it in `doctor`.
-    emit(["ready": true, "contextSize": SystemLanguageModel.default.contextSize])
+    // Guarded at 26.4, which is where `contextSize` became queryable — the
+    // enclosing block only promises 26.0, so an SDK between the two would fail
+    // to *compile* and the supervisor would simply not exist on that machine,
+    // reported as "could not build the local supervisor". A peer round is
+    // exactly where that costs somebody an afternoon, and the window is a
+    // nicety: it is printed in `doctor` and nothing depends on it.
+    var hello: [String: Any] = ["ready": true]
+    if #available(macOS 26.4, *) {
+        hello["contextSize"] = SystemLanguageModel.default.contextSize
+    }
+    emit(hello)
     while let line = readLine(strippingNewline: true) {
         if line.isEmpty { continue }
         guard let data = line.data(using: .utf8),
@@ -157,8 +213,11 @@ func serve() async {
         }
         let started = Date()
         do {
-            let session = LanguageModelSession(instructions: instructions)
-            let out = try await session.respond(to: prompt, generating: Judgement.self)
+            let mayAbstain = s.mayAbstain ?? false
+            let session = LanguageModelSession(instructions: mayAbstain ? abstainInstructions : instructions)
+            let decision: String = mayAbstain
+                ? try await session.respond(to: prompt, generating: CautiousJudgement.self).content.decision.rawValue
+                : try await session.respond(to: prompt, generating: Judgement.self).content.decision.rawValue
             // No reason field, deliberately. Asked for one it confabulated in
             // every observed run: a correct `stop` justified as "screen is
             // elsewhere" when the screen was exactly where the plan expected,
@@ -170,7 +229,7 @@ func serve() async {
             // one."* What the caller gets instead is which rule or which model
             // answered, which is true by construction.
             emit([
-                "decision": out.content.decision.rawValue,
+                "decision": decision,
                 "ms": Int(Date().timeIntervalSince(started) * 1000),
             ])
         } catch let err as LanguageModelSession.GenerationError {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "simframe",
-  "version": "0.12.2",
+  "version": "0.13.0",
   "mcpName": "io.github.lvlrSajjad/simframe",
   "description": "Always-warm iOS Simulator and Android emulator frames: agents read the screen in ~20ms instead of waiting on screenshots. MCP server + CLI.",
   "keywords": [

package/scripts/article-md.mjs

@@ -0,0 +1,185 @@
+#!/usr/bin/env node
+// Generate docs/ARTICLE.md from docs/agents-shouldnt-blink.html.
+//
+//   node scripts/article-md.mjs            # write it
+//   node scripts/article-md.mjs --check    # fail if it is out of date
+//
+// **Why generated rather than kept in step by hand.** The two were written as a
+// pair, with a note at the top of the Markdown saying it was the source of
+// record and to edit both together. Then the page grew several sections the
+// Markdown never got — the local supervisor, the capacity comparison — and the
+// note stayed there being wrong, which is worse than no note: the next person
+// follows it. Replacing a stale instruction with a stale *warning* would have
+// been the same mistake one level up.
+//
+// So the page is the source and this is the projection. Two copies of a
+// document cannot drift when one of them is derived, and `--check` in CI means
+// they cannot drift silently even for a commit.
+//
+// This is not a general HTML-to-Markdown converter and should not become one.
+// It understands exactly the constructs that article uses, and it *throws* on
+// anything it does not recognise rather than dropping it — a converter that
+// silently skips an unfamiliar tag produces a document that looks complete and
+// is not, which is the failure mode this whole file exists to prevent.
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+
+const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
+const SOURCE = path.join(ROOT, 'docs', 'agents-shouldnt-blink.html');
+const TARGET = path.join(ROOT, 'docs', 'ARTICLE.md');
+
+const ENTITIES = {
+  mdash: '—', ndash: '–', nbsp: ' ', amp: '&', lt: '<', gt: '>', quot: '"',
+  rsquo: '’', lsquo: '‘', ldquo: '“', rdquo: '”',
+  times: '×', rarr: '→', larr: '←', hellip: '…', plusmn: '±', deg: '°',
+  middot: '·', bull: '•', frac12: '½', le: '≤', ge: '≥', minus: '−', apos: "'",
+};
+
+/** Entities, then inline tags, then whitespace. Order matters. */
+function inline(html) {
+  let out = html
+    // The faculty spans open a paragraph and name the thing it defines —
+    // "Eyes", "Hand", "Memory" — with no separator in the HTML because the
+    // stylesheet supplies one.
+    .replace(/<span class="faculty">([\s\S]*?)<\/span>/g, (_, t) => `<strong>${t}</strong> — `)
+    // An abbreviation carries its expansion in the title attribute, which a
+    // reader of the Markdown has no way to hover over. Keep both.
+    .replace(/<abbr title="([^"]*)"[^>]*>([\s\S]*?)<\/abbr>/g, (_, full, t) => `${stripTags(t)} (${full})`)
+    .replace(/<a\b[^>]*href="([^"]*)"[^>]*>([\s\S]*?)<\/a>/g, (_, href, text) => `[${inline(text)}](${href})`)
+    .replace(/<code>([\s\S]*?)<\/code>/g, (_, t) => `\`${stripTags(t)}\``)
+    .replace(/<(?:b|strong)>([\s\S]*?)<\/(?:b|strong)>/g, (_, t) => `**${inline(t)}**`)
+    .replace(/<(?:i|em)>([\s\S]*?)<\/(?:i|em)>/g, (_, t) => `*${inline(t)}*`)
+    .replace(/<br\s*\/?>/g, '  \n');
+  const left = out.match(/<(?!\/)[a-z]/i);
+  if (left) throw new Error(`unhandled inline tag near: ${out.slice(Math.max(0, left.index - 40), left.index + 60)}`);
+  return decode(out).replace(/\s+/g, ' ').trim();
+}
+
+const decode = (s) => s
+  .replace(/&#(\d+);/g, (_, n) => String.fromCodePoint(Number(n)))
+  .replace(/&([a-zA-Z0-9]+);/g, (m, name) => {
+    if (!(name in ENTITIES)) throw new Error(`unknown entity &${name};`);
+    return ENTITIES[name];
+  });
+
+const stripTags = (s) => decode(s.replace(/<[^>]+>/g, '')).replace(/\s+/g, ' ').trim();
+
+/** A table, as GitHub-flavoured Markdown. */
+function table(html) {
+  const rows = [...html.matchAll(/<tr>([\s\S]*?)<\/tr>/g)].map((m) =>
+    [...m[1].matchAll(/<(th|td)\b[^>]*>([\s\S]*?)<\/\1>/g)].map((c) => inline(c[2])));
+  if (!rows.length) return [];
+  const head = /<thead>/.test(html) ? rows[0] : rows[0].map(() => ' ');
+  const body = /<thead>/.test(html) ? rows.slice(1) : rows;
+  const width = Math.max(...rows.map((r) => r.length));
+  const pad = (r) => [...r, ...Array(width - r.length).fill('')];
+  return [
+    `| ${pad(head).join(' | ')} |`,
+    `| ${Array(width).fill('---').join(' | ')} |`,
+    ...body.map((r) => `| ${pad(r).join(' | ')} |`),
+  ];
+}
+
+/** Everything inside one `<section>`, in order. */
+function blocks(html) {
+  const out = [];
+  // One pass over the constructs the article actually uses. `lastIndex`
+  // walking rather than nested parsing, because the document is flat inside a
+  // section and a real parser here would be a dependency.
+  const re = new RegExp([
+    '<h([123])>([\\s\\S]*?)<\\/h\\1>',
+    '<p class="sec-label">([\\s\\S]*?)<\\/p>',
+    '<p class="panel-label">([\\s\\S]*?)<\\/p>',
+    '<span class="amend-label">([\\s\\S]*?)<\\/span>',
+    '<p\\b[^>]*>([\\s\\S]*?)<\\/p>',
+    '<table>([\\s\\S]*?)<\\/table>',
+    '<pre>([\\s\\S]*?)<\\/pre>',
+    '<li>([\\s\\S]*?)<\\/li>',
+  ].join('|'), 'g');
+  let m;
+  while ((m = re.exec(html)) !== null) {
+    const [, level, heading, sec, panel, amend, para, tbl, pre, li] = m;
+    if (heading != null) out.push({ kind: 'h', level: Number(level), text: inline(heading) });
+    else if (sec != null) out.push({ kind: 'kicker', text: inline(sec) });
+    else if (panel != null) out.push({ kind: 'label', text: inline(panel) });
+    else if (amend != null) out.push({ kind: 'label', text: inline(amend) });
+    else if (para != null) out.push({ kind: 'p', text: inline(para) });
+    else if (tbl != null) out.push({ kind: 'table', lines: table(tbl) });
+    else if (pre != null) out.push({ kind: 'pre', text: decode(stripTagsPreserving(pre)) });
+    else if (li != null) out.push({ kind: 'li', text: inline(li) });
+  }
+  return out;
+}
+
+/** `<pre>` keeps its newlines; only tags come out. */
+const stripTagsPreserving = (s) => s.replace(/<[^>]+>/g, '').replace(/^\n/, '').replace(/\s+$/, '');
+
+export function render(html) {
+  const title = stripTags(html.match(/<h1>([\s\S]*?)<\/h1>/)[1]);
+  const kicker = stripTags(html.match(/<p class="kicker">([\s\S]*?)<\/p>/)[1]);
+  const standfirst = html.match(/<p class="standfirst">([\s\S]*?)<\/p>/);
+
+  const lines = [
+    `# ${title}`,
+    '',
+    `*${kicker}*`,
+    '',
+    '> **This file is generated** from [`agents-shouldnt-blink.html`](agents-shouldnt-blink.html)',
+    '> by `scripts/article-md.mjs`. Edit the page, not this — a hand-kept copy of a',
+    '> document is a copy that drifts, and this one did.',
+    '>',
+    '> The measurements, with N, median, p95 and the mistakes made getting to each,',
+    '> are in [`BENCHMARKS.md`](BENCHMARKS.md); what we expected before measuring is',
+    '> in [`EXPERIMENTS.md`](EXPERIMENTS.md).',
+    '',
+  ];
+  if (standfirst) lines.push(inline(standfirst[1]), '');
+  lines.push('---', '');
+
+  for (const section of html.match(/<section>[\s\S]*?<\/section>/g) ?? []) {
+    let pendingKicker = null;
+    let inList = false;
+    for (const b of blocks(section)) {
+      if (b.kind !== 'li' && inList) { lines.push(''); inList = false; }
+      switch (b.kind) {
+        // The kicker precedes its heading on the page and reads as a label for
+        // it, so it is held until the heading arrives rather than emitted where
+        // it was found.
+        case 'kicker': pendingKicker = b.text; break;
+        case 'h':
+          if (pendingKicker) { lines.push(`*${pendingKicker}*`, ''); pendingKicker = null; }
+          // `<h1>` is the title, already emitted above, so the page's `<h2>`
+          // sections are the document's second level and not its third.
+          lines.push(`${'#'.repeat(b.level)} ${b.text}`, '');
+          break;
+        case 'label': lines.push(`**${b.text}**`, ''); break;
+        case 'p': lines.push(b.text, ''); break;
+        case 'li': lines.push(`- ${b.text}`); inList = true; break;
+        case 'table': lines.push(...b.lines, ''); break;
+        case 'pre': lines.push('```', b.text, '```', ''); break;
+        default: throw new Error(`unhandled block ${b.kind}`);
+      }
+    }
+    if (inList) lines.push('');
+  }
+
+  return `${lines.join('\n').replace(/\n{3,}/g, '\n\n').trim()}\n`;
+}
+
+const html = fs.readFileSync(SOURCE, 'utf8');
+const md = render(html);
+
+if (process.argv.includes('--check')) {
+  const current = fs.existsSync(TARGET) ? fs.readFileSync(TARGET, 'utf8') : '';
+  if (current === md) {
+    console.log(`docs/ARTICLE.md is in step with the page (${md.split('\n').length} lines)`);
+    process.exit(0);
+  }
+  console.error('docs/ARTICLE.md is out of date with docs/agents-shouldnt-blink.html.');
+  console.error('Run: node scripts/article-md.mjs');
+  process.exit(1);
+}
+
+fs.writeFileSync(TARGET, md);
+console.log(`wrote docs/ARTICLE.md — ${md.split('\n').length} lines from ${SOURCE.split('/').pop()}`);

package/scripts/ci-integration-local.sh

@@ -43,6 +43,15 @@ for (const key of ["input.driver", "ax.driver"]) {
   console.log(`${good ? "ok  " : "FAIL"} ${key} = ${JSON.stringify(d[key])} (want "simframed")`);
   if (!good) failed = true;
 }
+// A configured driver is not an answering driver — the same check as ci.yml.
+// A whole CI run read the screen eighteen times, every reading came back
+// OCR-only, and this step said `ok` because a driver was present.
+{
+  const n = d["ax.elements"];
+  const good = typeof n === "number" && n > 0;
+  console.log(`${good ? "ok  " : "FAIL"} ax.elements = ${JSON.stringify(n)} (want > 0 — the tree must answer, not merely exist)`);
+  if (!good) failed = true;
+}
 if (d.warnings > 0) {
   console.log(`\n${d.warnings} degraded layer(s):`);
   for (const c of d.checks.filter((c) => c.level !== "ok")) console.log(`  ${c.level} ${c.name}: ${c.detail}`);
@@ -68,7 +77,19 @@ else
 fi
 
 step "The memory layer — screen map, refs, graph, verdicts, flows"
-node scripts/ci-memory.mjs --device="$DEVICE" && ok "ci-memory" || bad "ci-memory"
+# Mirrors the job: exit 75 means the display wedged and nothing was tested, so
+# revive once and run again; any other non-zero is a real check failing. The
+# wedge lands on an app switch and hit roughly every other run of this script on
+# the day it was written, which is most of why the job looked flaky.
+if node scripts/ci-memory.mjs --device="$DEVICE"; then
+  ok "ci-memory"
+elif [ $? = 75 ]; then
+  printf '     (the display wedged — DEFERRED 126. Reviving once and running again.)\n'
+  node src/cli.js revive --device="$DEVICE" || true
+  node scripts/ci-memory.mjs --device="$DEVICE" && ok "ci-memory (after one revive)" || bad "ci-memory"
+else
+  bad "ci-memory"
+fi
 
 step "The fingerprint distributions, measured and bounded"
 node scripts/eval-fingerprint.mjs --tour=test/tours/device-native.json --rounds=3 \

package/scripts/ci-memory.mjs

@@ -50,6 +50,9 @@ let failures = 0;
  */
 let deviceDied = null;
 
+/** EX_TEMPFAIL: the device died under the checks, so nothing was tested. */
+const DEVICE_DIED_EXIT = 75;
+
 function check(ok, label, detail = '') {
   if (!ok) failures += 1;
   console.log(`${ok ? 'ok  ' : 'FAIL'} ${label}${detail ? ` — ${detail}` : ''}`);
@@ -171,9 +174,21 @@ async function jsonRetry(args, opts, attempts = 3) {
     console.error(`  ${String(last.message).split('\n')[0]}`);
     console.error('\nEverything after this point would be testing a dead simulator, so the run');
     console.error('stops here. This is not a memory-layer failure — it is the device-state');
-    console.error('problem in docs/DEFERRED.md. A device restart is the only known cure;');
-    console.error('on a hosted runner it means a retry.');
-    process.exit(1);
+    console.error('problem in docs/DEFERRED.md (126). A device restart is the only known cure,');
+    console.error('and `simframe revive` is that restart.');
+    // Exit 75, not 1, and the distinction is the whole point of this file.
+    //
+    // "A check about the memory layer failed" and "the simulator died under the
+    // checks" are different conditions with different responses, and for two CI
+    // rounds they were one exit code — so a caller could only retry everything
+    // or retry nothing. 75 is EX_TEMPFAIL, which is exactly what this is: the
+    // subject under test was never reached.
+    //
+    // The caller reviving and running again is not papering over a product bug.
+    // The wedge is a documented CoreSimulator condition, `frame --fresh` names
+    // it, and `revive` is the cure this project ships for it — CI simply had no
+    // way to say "use it".
+    process.exit(DEVICE_DIED_EXIT);
   }
   throw last;
 }
@@ -244,7 +259,36 @@ if (failures) {
 
 console.log('\n--- the screen map ---');
 await jsonRetry(['do', LOOP], { allowFail: true });
-const map = await jsonRetry(['ui']);
+const map = await readableMap();
+
+/**
+ * A map that is empty *and* says a sensor failed is a blink, not a result.
+ *
+ * `jsonRetry` retries a command that throws, and this one did not throw: `ui`
+ * returned 200 with zero elements and a degraded note, which is the shape of
+ * failure this whole repo keeps writing rules about. The run that made this
+ * necessary had the accessibility read time out once, report an empty map, and
+ * then resolve a ref correctly sixty seconds later on the same device — so the
+ * device was fine and the check had caught one bad read.
+ *
+ * Deliberately narrow. An empty map with **no** degraded sensor is a real
+ * answer — that is a blank screen and the check should fail on it. Only an
+ * empty map that admits a layer did not answer is worth asking again, and after
+ * three attempts it fails with what the sensor said, which is the diagnosis
+ * either way.
+ */
+async function readableMap(attempts = 3) {
+  let last;
+  for (let i = 0; i < attempts; i += 1) {
+    last = await jsonRetry(['ui']);
+    if (last.elements?.length || !(last.degraded ?? []).length) return last;
+    if (i < attempts - 1) {
+      console.log(`     (the map came back empty and a sensor said why — retrying \`ui\`: ${(last.degraded ?? []).join('; ')})`);
+      await new Promise((r) => setTimeout(r, 2000));
+    }
+  }
+  return last;
+}
 
 // Report what actually answered rather than asserting the runner's situation.
 // This line used to read "with no accessibility tree available" unconditionally,
@@ -263,14 +307,38 @@ check(Number.isFinite(map.points?.width) && Number.isFinite(map.points?.height),
   'the map knows the screen size in points', `${map.points?.width}x${map.points?.height}pt`);
 
 const refs = (map.elements ?? []).map((e) => e.ref);
-check(refs.every((r, i) => r === i + 1),
-  'refs are numbered 1..n with no gaps', `#1..#${refs.length}`);
-check((map.elements ?? []).every((e) =>
-  Number.isInteger(e.x) && Number.isInteger(e.y)
-  && e.y >= 0 && e.y <= map.points.height && e.x >= 0 && e.x <= map.points.width),
-  'every element has a tap point on the screen');
-check(!(map.elements ?? []).some((e) => e.region === 'status-bar'),
-  'the status bar is not offered as something to tap');
+// Three checks over a collection, and `every`/`!some` are all true of an empty
+// one. On a CI run whose map came back with **0 elements** they printed
+// `ok   refs are numbered 1..n with no gaps — #1..#0` and two more like it:
+// three lines of reassurance about nothing, directly under the failure that
+// said the map was empty.
+//
+// That is the exact defect three field reports spent a day describing — a
+// confident statement that verified nothing — and the harness was doing it to
+// itself in the same output. Untested is not passed.
+if (!map.elements?.length) {
+  for (const label of [
+    'refs are numbered 1..n with no gaps',
+    'every element has a tap point on the screen',
+    'the status bar is not offered as something to tap',
+  ]) skip(label, 'the map was empty, so there was nothing to check');
+  // And say what the device was showing, because an empty map on a live device
+  // is the shape this project has chased under four different symptoms. The
+  // liveness note carries the ignored-gesture check; `stableForMs` and the
+  // frame age are the two numbers whose disagreement names a dead surface.
+  const st = await jsonRetry(['state'], { allowFail: true });
+  console.log(`     the device at that moment: frame #${st?.seq ?? '?'}, `
+    + `${st?.stableForMs ?? '?'}ms still, ${st?.live?.note ?? 'liveness reported nothing'}`);
+} else {
+  check(refs.every((r, i) => r === i + 1),
+    'refs are numbered 1..n with no gaps', `#1..#${refs.length}`);
+  check(map.elements.every((e) =>
+    Number.isInteger(e.x) && Number.isInteger(e.y)
+    && e.y >= 0 && e.y <= map.points.height && e.x >= 0 && e.x <= map.points.width),
+    'every element has a tap point on the screen');
+  check(!map.elements.some((e) => e.region === 'status-bar'),
+    'the status bar is not offered as something to tap');
+}
 
 console.log('\n--- element refs ---');
 // Re-read on a screen we chose, rather than on whatever the device happened to
@@ -296,7 +364,8 @@ if (first) {
   // hashes only have to agree with that, and they only get a say when they are
   // informative enough to have one.
   const moved = !informativeHash(before) || !informativeHash(after) || before !== after;
-  if (!ran(left)) skip('the screen actually changed before testing the stale ref',
+  const launched = ran(left);
+  if (!launched) skip('the screen actually changed before testing the stale ref',
     'the second app never launched, so there was no screen change to test against');
   else check(moved, 'the screen actually changed before testing the stale ref',
     `${before.slice(0, 10)} -> ${after.slice(0, 10)}`
@@ -305,7 +374,16 @@ if (first) {
   // reports "the stale-ref guard failed" for a device that never left the
   // screen, which is a false accusation against the one layer this file exists
   // to defend — and it is how this check has failed twice.
-  if (moved) {
+  // A skip has to propagate. The precondition above reported NOT TESTED and this
+  // check ran anyway and failed — which is the harness doing to itself, one line
+  // later, exactly what `skip` was written to stop it doing. `moved` is true
+  // when a hash is too degenerate to have a say, and that is right for "did the
+  // screen change" and wrong as a licence to run a check whose setup is known
+  // not to have happened.
+  if (!launched) {
+    skip('a ref numbered on another screen refuses instead of tapping those coordinates',
+      'we never reached another screen, so there was nothing to refuse from');
+  } else if (moved) {
     // This matched on prose twice and went red twice, both times for a refusal
     // that was correct and better worded than the alternation knew — most
     // recently `"Welcome to Reminders" is not on this screen`, which refuses