humanish 0.140.0 → 0.142.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (155) hide show
  1. package/ARCHITECTURE.md +1 -0
  2. package/CHANGELOG.md +23 -0
  3. package/README.md +2 -0
  4. package/dist/actors/codex/restricted-participant.js +1 -1
  5. package/dist/actors/computer-use/external-handoff.d.ts +22 -0
  6. package/dist/actors/computer-use/external-handoff.js +83 -0
  7. package/dist/actors/computer-use/loop/observation.d.ts +7 -0
  8. package/dist/actors/computer-use/loop/observation.js +17 -1
  9. package/dist/actors/computer-use/loop/types.d.ts +6 -0
  10. package/dist/analysis/admission.d.ts +8 -0
  11. package/dist/analysis/admission.js +15 -4
  12. package/dist/analysis/analysis-limits.d.ts +6 -0
  13. package/dist/analysis/analysis-limits.js +6 -0
  14. package/dist/analysis/analysis-lock.d.ts +8 -0
  15. package/dist/analysis/analysis-lock.js +129 -0
  16. package/dist/analysis/automatic-completion.js +7 -0
  17. package/dist/analysis/automatic-config.d.ts +5 -2
  18. package/dist/analysis/automatic-config.js +23 -8
  19. package/dist/analysis/automatic.js +13 -7
  20. package/dist/analysis/execute.d.ts +17 -6
  21. package/dist/analysis/execute.js +37 -19
  22. package/dist/analysis/gemini-wire.d.ts +8 -0
  23. package/dist/analysis/gemini-wire.js +165 -0
  24. package/dist/analysis/image-tokens.d.ts +4 -2
  25. package/dist/analysis/image-tokens.js +24 -3
  26. package/dist/analysis/job.d.ts +2 -0
  27. package/dist/analysis/job.js +7 -2
  28. package/dist/analysis/openai-wire.d.ts +5 -0
  29. package/dist/analysis/openai-wire.js +101 -0
  30. package/dist/analysis/provider.d.ts +15 -7
  31. package/dist/analysis/provider.js +21 -101
  32. package/dist/analysis/recording-ended.d.ts +16 -0
  33. package/dist/analysis/recording-ended.js +29 -0
  34. package/dist/analysis/responses.d.ts +3 -0
  35. package/dist/analysis/responses.js +19 -0
  36. package/dist/analysis/service.d.ts +1 -3
  37. package/dist/analysis/service.js +34 -53
  38. package/dist/analysis/source.d.ts +1 -1
  39. package/dist/analysis/types.d.ts +8 -2
  40. package/dist/analysis/validation.d.ts +1 -0
  41. package/dist/analysis/validation.js +13 -12
  42. package/dist/browser-control/protocol.d.ts +8 -0
  43. package/dist/browser-control/protocol.js +21 -0
  44. package/dist/cli/commands/analysis-signals.d.ts +10 -4
  45. package/dist/cli/commands/analysis-signals.js +28 -6
  46. package/dist/cli/commands/analyze.js +9 -7
  47. package/dist/cli/commands/observe.js +26 -1
  48. package/dist/cli/commands/run-command.d.ts +1 -1
  49. package/dist/cli/commands/run-command.js +12 -0
  50. package/dist/cli/commands/runs.js +8 -2
  51. package/dist/cli/commands/study-remote.d.ts +13 -0
  52. package/dist/cli/commands/study-remote.js +125 -0
  53. package/dist/cli/commands/study-route-computer-use.js +28 -4
  54. package/dist/cli/commands/study-route-scripted.js +3 -2
  55. package/dist/cli/commands/study-route-shared-world.js +3 -2
  56. package/dist/cli/commands/study-route-terminal.js +3 -2
  57. package/dist/cli/commands/study-run.js +34 -0
  58. package/dist/cli/commands/watch.js +5 -0
  59. package/dist/cli/doctor.js +7 -2
  60. package/dist/cli/findings.d.ts +30 -0
  61. package/dist/cli/findings.js +114 -15
  62. package/dist/cli/io.d.ts +8 -0
  63. package/dist/cli/io.js +3 -1
  64. package/dist/evidence/browser-hygiene.d.ts +12 -1
  65. package/dist/evidence/browser-hygiene.js +45 -2
  66. package/dist/feedback/export-bundle.js +1 -1
  67. package/dist/guest/external-links.d.ts +14 -0
  68. package/dist/guest/external-links.js +48 -0
  69. package/dist/guest/runtime-desktop.js +5 -2
  70. package/dist/keys/key-resolution.d.ts +10 -0
  71. package/dist/keys/key-resolution.js +33 -5
  72. package/dist/observer/data.d.ts +4 -0
  73. package/dist/observer/listen.d.ts +7 -2
  74. package/dist/observer/listen.js +10 -4
  75. package/dist/observer/render.d.ts +2 -0
  76. package/dist/observer/render.js +1 -1
  77. package/dist/observer/run-routes.js +12 -3
  78. package/dist/observer-app.html +2 -2
  79. package/dist/remote/launch.d.ts +53 -0
  80. package/dist/remote/launch.js +320 -0
  81. package/dist/remote/observer-proxy.d.ts +11 -0
  82. package/dist/remote/observer-proxy.js +66 -0
  83. package/dist/remote/plan.d.ts +35 -0
  84. package/dist/remote/plan.js +114 -0
  85. package/dist/remote/reattach.d.ts +17 -0
  86. package/dist/remote/reattach.js +81 -0
  87. package/dist/remote/runner.d.ts +68 -0
  88. package/dist/remote/runner.js +216 -0
  89. package/dist/routes/computer-use/e2b-desktop/desktop.js +27 -17
  90. package/dist/routes/computer-use/e2b-desktop/start.js +1 -1
  91. package/dist/routes/computer-use/e2b-desktop/teardown.js +1 -1
  92. package/dist/routes/computer-use/in-process-desktop.js +2 -2
  93. package/dist/routes/computer-use/live-phase.js +1 -1
  94. package/dist/routes/computer-use/setup.js +2 -2
  95. package/dist/routes/computer-use/types.d.ts +2 -0
  96. package/dist/routes/scripted/types.d.ts +2 -0
  97. package/dist/routes/shared-world/route.js +1 -1
  98. package/dist/routes/shared-world/types.d.ts +2 -0
  99. package/dist/routes/terminal/types.d.ts +2 -0
  100. package/dist/run/costs.d.ts +2 -1
  101. package/dist/run/display.d.ts +6 -1
  102. package/dist/run/display.js +11 -0
  103. package/dist/run/host-suspension.js +4 -1
  104. package/dist/run/preflight-receipts.d.ts +3 -12
  105. package/dist/run/preflight-receipts.js +7 -67
  106. package/dist/run/pricing.d.ts +13 -1
  107. package/dist/run/pricing.js +50 -4
  108. package/dist/run/process-owner.d.ts +28 -0
  109. package/dist/run/process-owner.js +83 -0
  110. package/dist/run/projection.js +1 -1
  111. package/dist/run/reclaim.d.ts +6 -0
  112. package/dist/run/reclaim.js +32 -2
  113. package/dist/run/results.d.ts +1 -1
  114. package/dist/run/route-shell.js +1 -1
  115. package/dist/run/run-cost.d.ts +2 -1
  116. package/dist/run/run-cost.js +6 -1
  117. package/dist/run/run-index.d.ts +3 -1
  118. package/dist/run/run-index.js +23 -11
  119. package/dist/run/run.d.ts +29 -5
  120. package/dist/run/run.js +43 -17
  121. package/dist/run/sandbox-creates.d.ts +10 -3
  122. package/dist/run/sandbox-creates.js +48 -6
  123. package/dist/run/status.d.ts +57 -2
  124. package/dist/run/status.js +87 -1
  125. package/dist/run/stored-runs.d.ts +3 -0
  126. package/dist/run/stored-runs.js +9 -4
  127. package/dist/run-study.d.ts +5 -0
  128. package/dist/study/doctor.js +14 -1
  129. package/dist/study/requirements.d.ts +1 -1
  130. package/dist/study/requirements.js +8 -6
  131. package/dist/study/route-inputs.js +4 -0
  132. package/dist/study/run-study-events.d.ts +8 -2
  133. package/dist/study/run-study-events.js +9 -2
  134. package/dist/substrates/e2b/base-module.d.ts +3 -0
  135. package/dist/substrates/e2b/base-module.js +40 -0
  136. package/dist/substrates/e2b/desktop-browser.d.ts +1 -1
  137. package/dist/substrates/e2b/desktop-browser.js +27 -7
  138. package/dist/substrates/e2b/desktop-executor.d.ts +7 -0
  139. package/dist/substrates/e2b/desktop-executor.js +3 -0
  140. package/dist/substrates/e2b/desktop-handoff.d.ts +22 -0
  141. package/dist/substrates/e2b/desktop-handoff.js +86 -0
  142. package/dist/substrates/e2b/sdk.d.ts +24 -0
  143. package/dist/substrates/e2b/sdk.js +5 -1
  144. package/dist/substrates/e2b/shell-only.d.ts +5 -0
  145. package/dist/substrates/e2b/shell-only.js +27 -0
  146. package/dist/substrates/local/runtime-release.js +8 -8
  147. package/dist/substrates/modal/sandbox.js +27 -28
  148. package/dist/substrates/modal/sdk.d.ts +7 -0
  149. package/dist/tui-app.js +115 -115
  150. package/docs/architecture/observer.md +7 -3
  151. package/docs/contracts/schemas.md +24 -3
  152. package/docs/contracts/study-analysis.md +52 -4
  153. package/docs/product/automatic-analysis.md +55 -9
  154. package/package.json +2 -2
  155. package/skills/humanish/SKILL.md +118 -5
@@ -159,9 +159,13 @@ allowlist, contained file reads, read-only routes and no-store security headers.
159
159
  Served data may also include `runtime` with `state`, `observedAt`, and
160
160
  `source: "local-run-status"`. This is a current observation of a contained,
161
161
  matching `status.json`; it never changes the run's recorded verdict or participant
162
- outcomes. A fresh heartbeat means running, an explicitly finalized record means
163
- finished, a record the CLI's signal handler wrote means interrupted, and stale or
164
- invalid timing means unknown. Missing, malformed, or mismatched records omit the
162
+ outcomes. A fresh heartbeat means running, a record its process wrote as `finished` means
163
+ finished, a record written as `interrupted` (by the CLI's signal handler, by a run that ended
164
+ without publishing, or by `humanish reclaim`) means interrupted, and stale or
165
+ invalid timing means unknown. While the run's own process analyzes it after the run ended,
166
+ `runtime.analysis` is `{ startedAt }`: the status band reads **Writing findings** and the
167
+ Findings view shows the analysis as pending, with how long ago it started, before the analysis
168
+ job exists. Missing, malformed, or mismatched records omit the
165
169
  observation. A stale heartbeat alone does not prove interruption, and stored PIDs
166
170
  are neither probed nor returned. Static rendering and export do not create this
167
171
  served-only observation.
@@ -1411,9 +1411,9 @@ proof claims zero on a `null` line, or when known spend exceeds the declared cap
1411
1411
 
1412
1412
  `humanish.run-status.v1` is `status.json`, written inside each run directory by
1413
1413
  every route at run start, refreshed on a fixed cadence while the run is
1414
- alive, and finalized when it ends: `{ schema, runId, state: running |
1414
+ alive, and rewritten when the run ends: `{ schema, runId, state: running |
1415
1415
  finished | interrupted, mode, study?, sandboxes?, pid, startedAt, updatedAt, completedAt?,
1416
- signal?, outcome? }`. A record written by 0.108 or earlier also has `lab`, with the same value as
1416
+ signal?, reason?, outcome?, analysis? }`. A record written by 0.108 or earlier also has `lab`, with the same value as
1417
1417
  `study`, and readers accept it. `sandboxes: none` is written at run start by a route that has no
1418
1418
  way to create a sandbox for the run: a scripted run against an `app-url` subject. A run that may
1419
1419
  create one has no `sandboxes` field. `humanish reclaim` reports such a run `clean` with
@@ -1453,6 +1453,24 @@ with it at publish: false means some participant or desktop usage has no price
1453
1453
  and the estimate is a lower bound. A record without it, as every record before
1454
1454
  0.109, says nothing about completeness, and readers treat it as unknown.
1455
1455
 
1456
+ `finished` means the run published its bundle. A process that leaves the run without publishing
1457
+ one, as a route that throws or returns a refusal after the run started does, writes
1458
+ `interrupted` with `reason: no-publish` and no `signal`; humanish 0.139.1 and earlier wrote
1459
+ `finished` with no `outcome` there. `humanish reclaim --run <id>` (not `--check`) rewrites a
1460
+ `running` record whose heartbeat is stale as `interrupted` with `reason: reclaimed` and
1461
+ `completedAt` set to its last `updatedAt`, and reports `markedInterrupted: true`. Either way the
1462
+ run is excluded from a study's duration and cost medians, which count only finished runs. Readers
1463
+ older than `reason` read such a record as interrupted.
1464
+
1465
+ `analysis` is added to a `finished` record when automatic analysis starts in the run's own
1466
+ process: `{ startedAt, endedAt?, state? }`. While it runs, the heartbeat refreshes `updatedAt`
1467
+ again; `completedAt` stays when the participants ended, so a run's duration excludes its
1468
+ analysis. When analysis ends, `endedAt` and `state` (the automatic analysis outcome's state:
1469
+ `complete`, `partial`, `failed`, `cancelled`, `skipped` or `unknown`) are written. An `analysis`
1470
+ without `endedAt` whose `updatedAt` is older than three touch intervals belonged to a process that
1471
+ stopped during analysis. `runAnalysisPhase` (`src/run/status.ts`) reads it as `running`,
1472
+ `stopped` or `ended`. Readers that predate the field ignore it.
1473
+
1456
1474
  `interrupted` with `signal` (`SIGINT`, `SIGTERM` or `SIGHUP`) is written by
1457
1475
  the CLI's run command when that signal stops a live run outside post-run
1458
1476
  analysis. It first prints one line saying it is stopping and refuses any further sandbox
@@ -1496,7 +1514,10 @@ cheapest-source-first: the `status.json` record, else the bundle, else the run
1496
1514
  directory alone. `{ runId, derivedFrom: status | bundle | directory, liveness,
1497
1515
  mode?, pid?, study?, startedAt?, updatedAt?, completedAt?, verdict?, ok?,
1498
1516
  failure?, participants?, estimatedCostUsd?, estimatedCostComplete?,
1499
- analysisCost?, durationMs? }`. `ok` and `failure` (the first execution failure)
1517
+ analysisCost?, durationMs?, analysis? }`. `analysis` is the status record's automatic analysis as
1518
+ `runAnalysisPhase` reads it, `{ state: running | stopped | ended, startedAt, endedAt?, outcome? }`;
1519
+ `humanish runs` and the TUI add `analyzing` or `analysis stopped` to the run's display from it, and
1520
+ `humanish runs --json` carries it on each run. `ok` and `failure` (the first execution failure)
1500
1521
  are the run's own outcome; `runDisplay` (`src/run/display.ts`) reads them with
1501
1522
  `liveness` and `verdict`, so the TUI and `humanish stats` show a run with
1502
1523
  `ok: false` as failed whatever its verdict. The point is cost: walking
@@ -5,7 +5,8 @@ It is separate from the participant's account, recorded outcome, and the run's
5
5
  deterministic review verdict. Opening an Observer never starts a provider request.
6
6
  Supported live runs request analysis on completion by default. A study that declares no analyst
7
7
  gets the Codex account analyst when its participant is a Codex `local-agent`, and the OpenAI
8
- analyst with a separate $10 cap on its expected cost otherwise. Set `review.analysis: false` to
8
+ analyst with a separate $10 cap on its expected cost otherwise. A study can also select the
9
+ [Gemini API analyst](#gemini-api-analysis). Set `review.analysis: false` to
9
10
  disable that request; see [automatic analysis](../product/automatic-analysis.md).
10
11
 
11
12
  ## Invocation
@@ -117,6 +118,49 @@ the current merge instructions, billed $1.92, most of its input served from the
117
118
  provider's prompt cache, and each of its 466 citations resolved. Two runs do not
118
119
  calibrate the merge request's estimate.
119
120
 
121
+ ## Gemini API analysis
122
+
123
+ `review.analysis.provider: gemini`, or `analyze --provider gemini`, sends the same request to
124
+ Google's Gemini API:
125
+
126
+ ```bash
127
+ humanish keys set gemini
128
+ humanish analyze --run latest --provider gemini --max-cost 2 --dry-run --json
129
+ humanish analyze --run latest --provider gemini --model gemini-3.8-flash --max-cost 2 --json
130
+ ```
131
+
132
+ The request goes to `generativelanguage.googleapis.com/v1beta/models/<model>:generateContent`
133
+ with the key in the `x-goog-api-key` header. It carries the same instructions, packet and
134
+ captures, with each capture at `MEDIA_RESOLUTION_HIGH`, thinking level `HIGH`, one candidate,
135
+ the standard service tier, no tools and `store: false`. The key is `GEMINI_API_KEY` when set,
136
+ else `GOOGLE_API_KEY`. Google's client libraries prefer `GOOGLE_API_KEY`; humanish does not, so a
137
+ `GOOGLE_API_KEY` exported for another Google API never replaces the stored Gemini key. Key
138
+ discovery fills neither name while the other is already in the environment. Without either, the attempt ends with
139
+ `ANALYSIS_API_KEY_MISSING` and nothing is sent; the OpenAI key is never used.
140
+
141
+ The supported model is `gemini-3.8-flash`, the one with a completed live analysis. The other
142
+ Gemini models whose pages list image input and structured output join once a live analysis on
143
+ each completes. `--max-cost` is required, and admission, the cost cap, `--max-output-tokens`
144
+ (at most 32,768) and the partial result for a bill above the worst case work as on OpenAI.
145
+ Admission prices each capture at 1,120 tokens, the media-resolution guide's figure for a
146
+ high-resolution image, and expects twice the output it expects from OpenAI: one
147
+ `gemini-3.8-flash` analysis of one participant wrote 24,246 output tokens. Usage maps
148
+ `promptTokenCount` to input, `candidatesTokenCount` plus `thoughtsTokenCount` to output, and
149
+ `cachedContentTokenCount` to cached input; a usage record whose `totalTokenCount` disagrees
150
+ with those parts is kept as unknown. Rates are Google's paid standard tier. `gemini-3.8-flash`
151
+ has a published price change on 2027-01-01, and each request is priced at the rates of the day
152
+ it is sent. A free-tier key bills nothing, and Google's terms
153
+ let it use free-tier content to improve its products.
154
+
155
+ Gemini refuses the analysis schemas while they carry `maxItems`, so the request leaves those
156
+ array limits out of the schema and states them, by field path, at the end of the system
157
+ instruction. The response is still checked against the full schema. A `STOP` candidate
158
+ whose only non-thought part is the JSON answer completes. `MAX_TOKENS` and `CONTINUATION` end
159
+ as `output_incomplete`. A blocked prompt and the `SAFETY`, `RECITATION`, `LANGUAGE`,
160
+ `BLOCKLIST`, `PROHIBITED_CONTENT`, `SPII`, `ESCALATION` and image-safety finish reasons end as
161
+ `refusal`. Any other finish reason or answer shape ends as `invalid_response`. An HTTP error
162
+ keeps only its status code, which the attempt's warnings name.
163
+
120
164
  ## Codex account analysis
121
165
 
122
166
  A study whose participant is a Codex `local-agent` uses this analyst unless it declares another,
@@ -424,7 +468,7 @@ It contains only the attempt/run IDs, input/config/source digests, prompt versio
424
468
  and timestamp. The live caller retains a binding to that exact directory and
425
469
  publishes the final receipt once. A start without usable final accounting means
426
470
  dispatch and spend remain unresolved; it is not proof that a provider charged.
427
- A cancellation before transport can finalize with `usage.dispatched: false`.
471
+ A cancellation before transport can end with `usage.dispatched: false`.
428
472
  Admission refusals, missing credentials and rejected dispatch guards do not
429
473
  create a potentially paid attempt. Reuse does not create a new start.
430
474
 
@@ -492,11 +536,15 @@ The states without findings:
492
536
  `analyze --max-cost 10`, which runs an analysis anyway. For a skip without `OPENAI_API_KEY`
493
537
  on a machine where Codex is signed in with a ChatGPT account (read when the view is built),
494
538
  `next` is `analyze --provider codex` and the message names `review.analysis.provider: codex`
495
- in the run's study file; otherwise the message says to set the key. For an analysis refused for its
539
+ in the run's study file; otherwise the message says to set the key. A study analyzed on Gemini
540
+ is skipped with `AUTOMATIC_ANALYSIS_GEMINI_KEY_MISSING`; its message names `keys set gemini`,
541
+ and `next` is `analyze --provider gemini --model <model> --max-cost 10`.
542
+ For an analysis refused for its
496
543
  cost, the message gives the expected cost, the worst case and the cap, and `next` sets
497
544
  `--max-cost` to the worst case rounded up.
498
545
  - `failed`: the latest attempt failed or was cancelled and no usable version exists. `next`
499
- reruns it with the same analyst, `--provider codex` for the Codex account analyst.
546
+ reruns it with the same analyst: `--provider codex` for the Codex account analyst, and
547
+ `--provider gemini --model <model>` for a Gemini one.
500
548
  - `none`: no analysis ran for this live run and nothing records why, for example a run that
501
549
  started from a library call, or one whose study file has moved or changed id.
502
550
  - `dry_run`: a dry run has no participant evidence to analyze. `next` is `run` with the run's own
@@ -50,7 +50,8 @@ allowance before deliberately choosing a larger budget. Explicit
50
50
  `maxOutputTokens` and `--max-output-tokens` limits are honored exactly. The output allowance
51
51
  includes reasoning as well as the report; exhausting it does not produce a usable
52
52
  report and never starts an automatic retry. Analysis
53
- sends selected retained text and captures to OpenAI using `OPENAI_API_KEY`.
53
+ sends selected retained text and captures to OpenAI using `OPENAI_API_KEY`, or to Google with
54
+ `provider: gemini`.
54
55
  Analysis runs in the humanish runner using its credentials. This setting adds no
55
56
  credential channel to the target application; each participant's route keeps
56
57
  its existing authentication boundary. Review the separate analysis budget before running a manifest live; an actor's
@@ -78,6 +79,27 @@ remain untested until a request. An omitted provider means this analyst for a Co
78
79
  `local-agent` participant on any target, and OpenAI for every other participant. This setting
79
80
  does not enable managed local desktops.
80
81
 
82
+ To use Google's Gemini API for the separate analyst:
83
+
84
+ ```yaml
85
+ review:
86
+ analysis:
87
+ provider: gemini
88
+ model: gemini-3.8-flash
89
+ maxCostUsd: 2
90
+ ```
91
+
92
+ Gemini analysis sends the same evidence packet and captures to Google, at high thinking level,
93
+ with each capture at high media resolution. It reads `GEMINI_API_KEY`, which `humanish keys set
94
+ gemini` stores, and `GOOGLE_API_KEY` only when `GEMINI_API_KEY` is unset. Key discovery treats
95
+ the two names as one key: a project's `.humanish/local/provider.env` or the user store never adds
96
+ one while the other is already in the environment. A missing Gemini key skips the analysis
97
+ (`AUTOMATIC_ANALYSIS_GEMINI_KEY_MISSING`); the OpenAI key is never used in its place. Admission,
98
+ `maxCostUsd`, `maxOutputTokens` and priced usage work as they do on OpenAI, at the paid-tier
99
+ rates in `src/run/pricing.ts`; requests made with a free-tier key bill nothing, and Google's
100
+ terms let it use free-tier content to improve its products. The supported model is
101
+ `gemini-3.8-flash`, the one with a completed live analysis.
102
+
81
103
  The same configuration works through `humanish run <study>`,
82
104
  `watch <study>`, and TUI live starts. Direct library calls to the five recording
83
105
  producers honor it too. Supported routes are computer-use, scripted-browser,
@@ -86,13 +108,34 @@ never enables analysis by default and rejects an explicit analysis mapping
86
108
  before execution. `false` is accepted on every route. Dry runs show analysis
87
109
  as skipped, without reading analysis credentials or making a provider request.
88
110
 
89
- Participant execution finishes and its recording is finalized before analysis is
90
- queued. A participant who was blocked or interrupted can still have useful
91
- retained evidence; analysis requires a verified live recording, not a successful
92
- participant outcome. An active, missing or invalid recording is not analyzed.
93
- Default analysis also skips recordings containing only setup or failure records
94
- with no retained participant activity. A desktop startup failure does not start
95
- an analysis request. The original failure remains visible.
111
+ Automatic analysis starts when the run ends: every participant has stopped and the run has
112
+ published its bundle (`run.json` with an outcome, and `status.json` state `finished`). It runs in
113
+ the same process, so `humanish run` returns only after it. A participant who was blocked or
114
+ interrupted can still have useful retained evidence; analysis requires a verified live recording
115
+ whose participant streams all ended, not a successful participant outcome. An active, missing or
116
+ invalid recording is not analyzed. Default analysis also skips recordings containing only setup
117
+ or failure records with no retained participant activity. A desktop startup failure does not
118
+ start an analysis request. The original failure remains visible.
119
+
120
+ While analysis runs, the run reads as analyzing. `status.json` records `analysis.startedAt` and
121
+ its heartbeat keeps refreshing `updatedAt`; `humanish runs` lists the run as `passed, analyzing`
122
+ (or its other result); the TUI adds `analyzing` to the run; the served Observer shows **Writing
123
+ findings** in its status band and the Findings tab in a pending state with how long ago analysis
124
+ started; `humanish review` says the analysis is running and when it started. In a terminal, `run`
125
+ prints `Participants finished. Writing findings; this usually takes 1 to 4 minutes.` to stderr,
126
+ then `Still writing findings: <elapsed> elapsed.` every 20 seconds. With `--json` it prints only
127
+ the first line, on stderr.
128
+
129
+ A run whose process ended before it published (a crash, or a refusal after the run started) is
130
+ recorded `interrupted` with `reason: no-publish`, and one whose process was killed is marked
131
+ `interrupted` by `humanish reclaim`. `humanish analyze` accepts an interrupted run when every
132
+ participant stream ended; when one was still running it refuses with
133
+ `ANALYSIS_RUN_INCOMPLETE` and names `humanish run <study>`. It refuses a run that is still alive
134
+ with `ANALYSIS_RUN_ACTIVE`, and a run whose heartbeat stopped while its pid is still in use with
135
+ `ANALYSIS_RUN_UNCONFIRMED`, naming `humanish reclaim --run <id>`. When the process running an
136
+ automatic analysis stops before it finishes, `review` says so and names the `analyze` command
137
+ that runs it again. That command takes over the run's `.analysis-lock` when the process that
138
+ held it is gone, and says so in a warning.
96
139
 
97
140
  CLI live starts disclose the selected analyst, its expected cost range and cap, or unknown account dollars before execution.
98
141
  `humanish study check <study> --json` and the TUI study screen also expose the
@@ -126,7 +169,8 @@ run's own result is ok, the first line of the human output reads
126
169
  command: with Codex signed in with a ChatGPT account it is `humanish analyze --run <id>
127
170
  --provider codex`, and the message names `review.analysis.provider: codex` as the setting that
128
171
  makes that analyst the study's; otherwise it says to set the key and names `humanish analyze
129
- --run <id> --max-cost 10`. So does a
172
+ --run <id> --max-cost 10`. A study analyzed on Gemini names `humanish keys set gemini` and
173
+ `humanish analyze --run <id> --provider gemini --model <model> --max-cost 10`. So does a
130
174
  default analysis whose expected cost, plus the margin, is over the default $10 cap:
131
175
  it is skipped with `AUTOMATIC_ANALYSIS_ADMISSION_REFUSED` and the run exits 0. For
132
176
  any analysis refused for its cost, the job records the expected cost, the worst
@@ -176,6 +220,7 @@ JSON also has `findings`:
176
220
  | `AUTOMATIC_ANALYSIS_SOURCE_UNAVAILABLE` | `skipped` | This invocation published no run to analyze, or the run id is invalid |
177
221
  | `AUTOMATIC_ANALYSIS_SOURCE_CHANGED` | `failed` | The run directory changed after the run published its bundle |
178
222
  | `AUTOMATIC_ANALYSIS_KEY_MISSING` | `skipped` | No OpenAI API key |
223
+ | `AUTOMATIC_ANALYSIS_GEMINI_KEY_MISSING` | `skipped` | No Gemini API key for a study analyzed on Gemini |
179
224
  | `AUTOMATIC_ANALYSIS_NO_PARTICIPANT_EVIDENCE` | `skipped` | A default analysis found no participant evidence |
180
225
  | `AUTOMATIC_ANALYSIS_ACTOR_CANCELLED` | `skipped` | The harness cancelled the participants, or a signal stopped the run |
181
226
  | `AUTOMATIC_ANALYSIS_CLEANUP_UNCONFIRMED` | `skipped` | A local VM study could not confirm its cleanup; the command exits 2 |
@@ -187,6 +232,7 @@ JSON also has `findings`:
187
232
  | `AUTOMATIC_ANALYSIS_PUBLICATION_FAILED` | `failed` | The analysis could not be published |
188
233
  | `AUTOMATIC_ANALYSIS_CANCELLATION_UNAVAILABLE` | `unknown` | A cancellation could not be recorded |
189
234
  | `AUTOMATIC_ANALYSIS_OUTCOME_UNKNOWN` | `unknown` | The outcome could not be determined |
235
+ | `AUTOMATIC_ANALYSIS_STOPPED` | `unknown` | Read only: the process running it stopped refreshing its job record |
190
236
 
191
237
  A run refused before it was created (`run: not-created`) prints no `analysis:` line, since there
192
238
  is no run to analyze and the refusal says why. Its JSON keeps the record, with
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "humanish",
3
- "version": "0.140.0",
3
+ "version": "0.142.0",
4
4
  "description": "Synthetic user research for apps, CLIs, and agent-facing product flows.",
5
5
  "keywords": [
6
6
  "cli",
@@ -96,7 +96,7 @@
96
96
  "comms:agentmail:contract": "node --import tsx scripts/agentmail-contract-check.mjs",
97
97
  "format": "oxfmt",
98
98
  "format:check": "oxfmt --check",
99
- "lint": "node scripts/check-lint-cap.mjs --max-warnings=411",
99
+ "lint": "node scripts/check-lint-cap.mjs --max-warnings=410",
100
100
  "knip": "knip",
101
101
  "prose:check": "node scripts/check-code-prose.mjs",
102
102
  "vocabulary:check": "tsx scripts/check-retired-words.ts",
@@ -108,6 +108,16 @@ not local inference or the participant's existing conversation. See
108
108
  [the analysis contract](../../docs/contracts/study-analysis.md) for the current
109
109
  CLI/model qualification and setup limits.
110
110
 
111
+ To analyze on Gemini, the user stores a key with `npx humanish keys set gemini`
112
+ (`GEMINI_API_KEY`; `GOOGLE_API_KEY` is read only when `GEMINI_API_KEY` is
113
+ unset), then sets `review.analysis: {provider: gemini, maxCostUsd: <n>}` or
114
+ passes `analyze --provider gemini --max-cost <n>`. The supported model is
115
+ `gemini-3.8-flash`. This sends the evidence packet and
116
+ captures to Google. Admission, the cost cap and priced usage work as on OpenAI,
117
+ and a missing Gemini key never falls back to OpenAI. On a free-tier key, Google
118
+ may use the content to improve its products; tell the user before a study with
119
+ private evidence runs on one.
120
+
111
121
  When post-run analysis is refused for its cost, the run output and `humanish
112
122
  review` give its expected cost, its worst case and the command that runs it,
113
123
  `humanish analyze --run <id> --max-cost <n>`. Show the user those costs and run
@@ -298,6 +308,63 @@ fails with `HUMANISH_STUDY_RETIRED_DIRECTORY`; move it to the matching `studies/
298
308
  run looks like idle tiles — another reason to leave the cap out unless you
299
309
  need it.
300
310
 
311
+ ### Phone, tablet and desktop in one study
312
+
313
+ When the user asks to test several screen sizes, write one study with one
314
+ `participants` entry per size and set `device` on each entry. The participants run
315
+ at once, the Observer shows them side by side, and the findings compare them.
316
+
317
+ ```yaml
318
+ schema: humanish.study.v3
319
+ id: devices
320
+ title: Phone, tablet and desktop
321
+ route: computer-use
322
+ mode: live
323
+ subject:
324
+ source: app-url
325
+ appUrl: "http://127.0.0.1:3000"
326
+ actor:
327
+ type: local-agent
328
+ localAgent: codex
329
+ persona: synthetic-new-user
330
+ mission: "Sign up and create your first project. Say what was hard to read or reach."
331
+ participants:
332
+ - id: phone
333
+ device: mobile
334
+ - id: tablet
335
+ device: tablet
336
+ - id: desktop
337
+ device: desktop
338
+ execution:
339
+ target: local
340
+ timeoutMs: 600000
341
+ desktop:
342
+ fidelity:
343
+ mobileEmulation: true
344
+ ```
345
+
346
+ This runs on the local browser runtime with the user's Codex login. For a deployed
347
+ app, set `execution.target: e2b-desktop`, put the public URL in `subject.appUrl`,
348
+ add `policies.allowPublicTargets: true`, and give each entry a `target` (below).
349
+
350
+ - `device` takes a preset: `mobile` (414×896), `small-mobile`, `narrow-mobile`,
351
+ `tablet` (820×1180), `desktop` (1440×950) or `wide` (1920×1080). The preset
352
+ sizes the participant's screen and browser window.
353
+ - `fidelity.mobileEmulation: true` gives the phone participants a mobile viewport,
354
+ device pixel ratio, touch and a mobile user agent in Chromium. Tablet and
355
+ desktop participants are left as they are.
356
+ - Do not run one study per size, draw a device frame inside the page, or resize
357
+ the browser from the mission. Those produce desktop-ratio evidence labelled as
358
+ a phone, and three runs the Observer cannot compare.
359
+ - Participants on one public app (`policies.allowPublicTargets: true`) share its
360
+ data, so more than one is refused unless each entry sets `target`: a separate
361
+ test account, tenant or seeded URL per participant, or the same URL when the
362
+ task has each participant create its own account. A loopback or clone subject
363
+ needs no `target`.
364
+ - On a local study, `doctor --study` fails its runtime row when the installed
365
+ local runtime predates per-participant screens; `humanish runtime setup`
366
+ installs the pinned one. Do not move the study to E2B without asking the user.
367
+
301
368
  Use committed `humanish/studies/*.yaml` for public-safe, reproducible studies. Use
302
369
  ignored `.humanish/studies/*.yaml` or `.humanish/local/studies/*.yaml` for private repo
303
370
  targets, local-only dogfood, or machine-specific settings. Never commit private
@@ -472,6 +539,26 @@ Label the participant's own report in the review summary as the participant's ac
472
539
  place of findings; `analysis.next` is null when no command can produce them. `latest` moves with
473
540
  every run, including dry runs, so use the run id the run printed.
474
541
 
542
+ ## Long Studies: `run` Blocks Until Its Findings
543
+
544
+ `humanish run <study>` returns only after every participant has ended and the automatic analysis
545
+ has finished. A participant with `endAfterMs` can keep a study running for up to 24 hours, and
546
+ analysis usually adds 1 to 4 minutes after the last participant ends. A shell timeout that sends
547
+ SIGTERM stops the run: humanish marks it interrupted and stops its sandboxes.
548
+
549
+ - Start a study that can outlive your shell call in tmux, for example
550
+ `tmux new -d -s study 'npx humanish run <study> --json --no-open > run-result.json 2> run.log'`.
551
+ `--detach` only skips the Observer server at the end; it does not move the run to the background.
552
+ - The first stderr line names the run: `Run <id> started. Follow it with humanish observe --run <id>`.
553
+ If you did not keep it, `npx humanish runs --json` lists runs newest first.
554
+ - In `runs --json`, a run whose participants are working has `display.state: running`; once they
555
+ end, its `display` is their result and `analysis.state` is `running` while findings are written.
556
+ - `npx humanish review --run <id> --json` says where the run is: participants running, analysis
557
+ running since a given time, or the findings.
558
+ - If the run's process died, `npx humanish reclaim --run <id>` stops its sandboxes and marks it
559
+ interrupted. `npx humanish analyze --run <id>` then analyzes it if every participant had ended,
560
+ and otherwise names the command that runs the study again.
561
+
475
562
  ## Optional Live E2B Study
476
563
 
477
564
  Live headed E2B desktop participants are optional. Add the substrate dependency only
@@ -481,12 +568,18 @@ when the user explicitly wants live E2B execution:
481
568
  npm i -D @e2b/desktop
482
569
  ```
483
570
 
484
- Then confirm env var names are documented without values:
571
+ Then confirm env var names are documented without values. Which keys a hosted
572
+ study needs depends on its participant and analyst:
485
573
 
486
- ```bash
487
- E2B_API_KEY
488
- OPENAI_API_KEY
489
- ```
574
+ | Participant | Keys |
575
+ | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
576
+ | `openai-computer-use` | `E2B_API_KEY`, `OPENAI_API_KEY` |
577
+ | `local-agent` with `localAgent: codex` | `E2B_API_KEY` and `codex login`; the findings analysis uses the same Codex account |
578
+ | `local-agent` with `localAgent: claude` | `E2B_API_KEY` and `claude auth login`; `OPENAI_API_KEY` for findings; without it, a signed-in Codex can analyze the run (the run prints the command) |
579
+
580
+ A hosted Codex participant does not need `OPENAI_API_KEY`. Do not report a missing
581
+ `OPENAI_API_KEY` as the reason a Codex study on E2B fails; read the
582
+ `doctor --study` row instead.
490
583
 
491
584
  Do not paste values into files, prompts, run bundles, issue drafts, or logs.
492
585
  Load local values only at invocation time:
@@ -502,6 +595,26 @@ declared scenario is developer-experience testing. Private repos are allowed
502
595
  only as explicit maintainer-authorized runs with repo redaction left on; never
503
596
  publish their names, screenshots, logs, source snippets, or operational details.
504
597
 
598
+ ## Setup errors and what they mean
599
+
600
+ Run `npx humanish doctor --study <id>` before the first live run and read every
601
+ `missing` row. The common ones:
602
+
603
+ - `codex_unsafe_configuration` on a hosted Codex participant: the row names the
604
+ setting humanish refused. Report it to the user; never edit, move or rename
605
+ their Codex, agent or skill config to make a run pass. An `AGENTS.md` in the
606
+ Codex home is admitted, and doctor notes that the participant also receives it.
607
+ - `subject.appUrl` must be loopback: a non-loopback URL needs
608
+ `policies.allowPublicTargets: true`, and with more than one participant, a
609
+ `target` on each entry.
610
+ - "Will be skipped: OPENAI_API_KEY is missing" on `post-run analysis` (an
611
+ `openai-computer-use` or Claude participant): the run works without findings,
612
+ and when Codex is signed in, its `next:` line gives the command that analyzes it with Codex.
613
+ `review.analysis.provider: codex` makes that the study's default.
614
+ - A local browser study refuses a non-loopback URL, a port below 1024, hosted
615
+ templates and `sandboxTimeoutMs`. The message names the setting; do not switch the
616
+ study to a hosted target unless the user agrees to E2B usage.
617
+
505
618
  ## Reporting Back
506
619
 
507
620
  Report a live run's findings first, as