humanish 0.110.0 → 0.110.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (178) hide show
  1. package/ARCHITECTURE.md +15 -12
  2. package/CHANGELOG.md +26 -0
  3. package/CONTEXT.md +8 -6
  4. package/README.md +103 -53
  5. package/dist/actors/computer-use/loop/ending.js +4 -4
  6. package/dist/actors/local-agent/claude-participant.d.ts +74 -0
  7. package/dist/actors/local-agent/claude-participant.js +243 -0
  8. package/dist/actors/local-agent/claude-session.d.ts +16 -3
  9. package/dist/actors/local-agent/claude-session.js +64 -23
  10. package/dist/actors/local-agent/cli.d.ts +17 -0
  11. package/dist/actors/local-agent/cli.js +111 -24
  12. package/dist/actors/local-agent/readiness.d.ts +2 -2
  13. package/dist/actors/local-agent/readiness.js +20 -12
  14. package/dist/cli/commands/analyze.js +21 -5
  15. package/dist/cli/commands/observe.js +15 -2
  16. package/dist/cli/commands/run-signals.d.ts +2 -0
  17. package/dist/cli/commands/run-signals.js +106 -45
  18. package/dist/cli/commands/runs.d.ts +5 -0
  19. package/dist/cli/commands/runs.js +113 -74
  20. package/dist/cli/commands/setup.js +23 -14
  21. package/dist/cli/commands/study-route-computer-use.js +9 -3
  22. package/dist/cli/commands/study-route-open.js +1 -0
  23. package/dist/cli/commands/study-route-preview.js +1 -0
  24. package/dist/cli/commands/study-route-scripted.js +2 -0
  25. package/dist/cli/commands/study-route-shared-world.js +6 -2
  26. package/dist/cli/commands/study-route-terminal.js +2 -0
  27. package/dist/cli/commands/watch.js +14 -4
  28. package/dist/cli/doctor.d.ts +4 -1
  29. package/dist/cli/doctor.js +51 -10
  30. package/dist/cli/findings.d.ts +103 -0
  31. package/dist/cli/findings.js +317 -0
  32. package/dist/cli/first-run-path.d.ts +32 -11
  33. package/dist/cli/first-run-path.js +113 -69
  34. package/dist/cli/invocation.d.ts +7 -0
  35. package/dist/cli/invocation.js +32 -0
  36. package/dist/cli/io.d.ts +3 -1
  37. package/dist/cli/io.js +8 -15
  38. package/dist/cli/observer-follow.d.ts +20 -0
  39. package/dist/cli/observer-follow.js +36 -1
  40. package/dist/cli/orientation.js +16 -14
  41. package/dist/cli/product-sentence.d.ts +3 -3
  42. package/dist/cli/product-sentence.js +3 -3
  43. package/dist/cli/version.d.ts +1 -0
  44. package/dist/cli/version.js +11 -0
  45. package/dist/evidence/encoded-text.d.ts +9 -6
  46. package/dist/evidence/encoded-text.js +84 -19
  47. package/dist/evidence/redaction.d.ts +2 -0
  48. package/dist/evidence/redaction.js +123 -21
  49. package/dist/feedback/export-bundle.js +2 -2
  50. package/dist/keys/env-file.d.ts +4 -0
  51. package/dist/keys/env-file.js +17 -0
  52. package/dist/keys/key-resolution.js +6 -5
  53. package/dist/observer/data.d.ts +14 -1
  54. package/dist/observer/data.js +19 -9
  55. package/dist/observer/exposure.js +1 -1
  56. package/dist/observer/library.d.ts +4 -0
  57. package/dist/observer/library.js +21 -17
  58. package/dist/observer/render.js +10 -3
  59. package/dist/observer/run-routes.d.ts +2 -0
  60. package/dist/observer/run-routes.js +51 -35
  61. package/dist/observer/serve.js +2 -1
  62. package/dist/observer-app.html +3 -3
  63. package/dist/routes/computer-use/bundle.js +3 -1
  64. package/dist/routes/computer-use/fanout-records.js +1 -1
  65. package/dist/routes/computer-use/participant-execution.js +3 -1
  66. package/dist/routes/computer-use/participant-model.js +5 -2
  67. package/dist/routes/computer-use/plan.js +9 -4
  68. package/dist/routes/computer-use/preflight.js +1 -1
  69. package/dist/routes/computer-use/result.js +17 -28
  70. package/dist/routes/computer-use/single-bundle.js +7 -1
  71. package/dist/routes/scripted/bundle.js +4 -2
  72. package/dist/routes/scripted/plan.js +2 -2
  73. package/dist/routes/scripted/result.d.ts +3 -5
  74. package/dist/routes/scripted/result.js +16 -22
  75. package/dist/routes/scripted/scenario.js +4 -4
  76. package/dist/routes/shared-world/bundle-records.js +6 -1
  77. package/dist/routes/shared-world/bundle.js +2 -0
  78. package/dist/routes/shared-world/plan.js +6 -2
  79. package/dist/routes/shared-world/result.js +22 -22
  80. package/dist/routes/terminal/bundle.js +6 -3
  81. package/dist/routes/terminal/credentials.d.ts +5 -5
  82. package/dist/routes/terminal/credentials.js +14 -6
  83. package/dist/routes/terminal/dry-run.js +8 -12
  84. package/dist/routes/terminal/ledger.d.ts +3 -3
  85. package/dist/routes/terminal/ledger.js +6 -6
  86. package/dist/routes/terminal/live-finish.js +15 -13
  87. package/dist/routes/terminal/plan.js +12 -4
  88. package/dist/routes/terminal/result.d.ts +6 -7
  89. package/dist/routes/terminal/result.js +6 -22
  90. package/dist/routes/terminal/route.js +5 -4
  91. package/dist/routes/terminal/runtime-auth.d.ts +6 -0
  92. package/dist/routes/terminal/runtime-auth.js +5 -0
  93. package/dist/routes/terminal/session.js +1 -1
  94. package/dist/run/bundle-shape.d.ts +3 -1
  95. package/dist/run/bundle-shape.js +25 -1
  96. package/dist/run/bundle.d.ts +24 -0
  97. package/dist/run/display.d.ts +59 -0
  98. package/dist/run/display.js +106 -0
  99. package/dist/run/dry-run.js +13 -8
  100. package/dist/run/judge.d.ts +3 -2
  101. package/dist/run/judge.js +12 -1
  102. package/dist/run/local-only-files.d.ts +13 -0
  103. package/dist/run/local-only-files.js +35 -0
  104. package/dist/run/pricing.js +1 -1
  105. package/dist/run/projection.d.ts +0 -2
  106. package/dist/run/projection.js +0 -11
  107. package/dist/run/reclaim-outcomes.d.ts +89 -0
  108. package/dist/run/reclaim-outcomes.js +213 -0
  109. package/dist/run/reclaim.d.ts +23 -17
  110. package/dist/run/reclaim.js +213 -114
  111. package/dist/run/results.d.ts +10 -20
  112. package/dist/run/results.js +2 -2
  113. package/dist/run/run-index.d.ts +11 -1
  114. package/dist/run/run-index.js +11 -0
  115. package/dist/run/run-not-found.js +3 -2
  116. package/dist/run/run.d.ts +34 -18
  117. package/dist/run/run.js +166 -67
  118. package/dist/run/sandbox-creates.d.ts +59 -0
  119. package/dist/run/sandbox-creates.js +140 -0
  120. package/dist/run/sandbox-receipts.d.ts +20 -5
  121. package/dist/run/sandbox-receipts.js +56 -7
  122. package/dist/run/stats.d.ts +5 -0
  123. package/dist/run/stats.js +8 -4
  124. package/dist/run/status.d.ts +6 -9
  125. package/dist/run/status.js +7 -23
  126. package/dist/run/stored-runs.d.ts +5 -14
  127. package/dist/run/stored-runs.js +31 -146
  128. package/dist/run/streams.d.ts +3 -2
  129. package/dist/study/agents-section.d.ts +21 -0
  130. package/dist/study/agents-section.js +86 -0
  131. package/dist/study/doctor.d.ts +1 -1
  132. package/dist/study/doctor.js +8 -4
  133. package/dist/study/init-templates.js +0 -95
  134. package/dist/study/init.js +45 -37
  135. package/dist/study/keys.d.ts +12 -0
  136. package/dist/study/keys.js +32 -0
  137. package/dist/study/local-browser-host.d.ts +11 -0
  138. package/dist/study/local-browser-host.js +42 -0
  139. package/dist/study/parse/actors.js +9 -1
  140. package/dist/study/parse/subject.js +12 -0
  141. package/dist/study/parse/url-credentials.d.ts +9 -0
  142. package/dist/study/parse/url-credentials.js +50 -0
  143. package/dist/study/persona.d.ts +4 -0
  144. package/dist/study/persona.js +4 -2
  145. package/dist/study/preflight-probes.js +2 -1
  146. package/dist/study/types.d.ts +8 -6
  147. package/dist/study/url-credentials.d.ts +6 -0
  148. package/dist/study/url-credentials.js +34 -0
  149. package/dist/substrates/e2b/peer-install.d.ts +51 -0
  150. package/dist/substrates/e2b/peer-install.js +161 -0
  151. package/dist/substrates/e2b/sandbox.d.ts +34 -6
  152. package/dist/substrates/e2b/sandbox.js +141 -30
  153. package/dist/substrates/e2b/sdk.d.ts +33 -0
  154. package/dist/substrates/e2b/sdk.js +21 -21
  155. package/dist/substrates/local/firecracker-desktop.js +3 -2
  156. package/dist/substrates/local/runtime-host.js +5 -4
  157. package/dist/substrates/local/runtime.js +5 -4
  158. package/dist/substrates/shell.d.ts +2 -0
  159. package/dist/substrates/shell.js +4 -0
  160. package/dist/tui/contract.js +5 -4
  161. package/dist/tui-app.js +133 -133
  162. package/dist/verify/liveness.d.ts +19 -3
  163. package/dist/verify/liveness.js +47 -25
  164. package/dist/verify/verify.d.ts +3 -0
  165. package/dist/verify/verify.js +8 -3
  166. package/docs/architecture/actor-contract.md +2 -1
  167. package/docs/architecture/observer.md +22 -7
  168. package/docs/architecture/task-protocol-support.md +13 -13
  169. package/docs/architecture/terminal-product-route.md +15 -13
  170. package/docs/contracts/run-bundle.md +95 -10
  171. package/docs/contracts/schemas.md +44 -26
  172. package/docs/contracts/study-analysis.md +45 -0
  173. package/docs/contracts/study-costs.md +8 -0
  174. package/docs/decisions/0001-run-bundle-is-the-source-of-truth.md +3 -0
  175. package/docs/decisions/0003-contained-paths-and-cleanup-authority.md +9 -3
  176. package/docs/principles/invariants-and-defaults.md +11 -8
  177. package/package.json +6 -3
  178. package/skills/humanish/SKILL.md +38 -20
@@ -37,6 +37,7 @@ workflow without leaking private upstream truth into core.
37
37
  | Review | `humanish.review.v1` | `contract-proof-review` |
38
38
  | Study analysis | `humanish.study-analysis.v1` | see [study analysis](study-analysis.md) and synthetic analysis fixtures in `tests/` |
39
39
  | Study analysis correction | `humanish.study-analysis-correction.v1` | see [study analysis](study-analysis.md#human-review-and-sharing) |
40
+ | Analysis findings view | `humanish.analysis-findings.v1` (`review --json` field `analysis`; read-only projection of the selected analysis) | see [study analysis](study-analysis.md#reading-findings) |
40
41
  | Analysis execution receipt | `humanish.analysis-execution.v1` | see [study analysis](study-analysis.md#durable-records) |
41
42
  | Analysis execution start | `humanish.analysis-execution-start.v1` | see [study analysis](study-analysis.md#durable-records) |
42
43
  | Verification | `humanish.verify-result.v1` | `five-check-verify` |
@@ -278,7 +279,7 @@ instruction?, target?, entry?, host?, reasoningEffort?, stopWhen?, dwell? }` bec
278
279
  an interactive PTY would overstate the mechanism (a duplex PTY
279
280
  transport does not ship). `terminal.stdin` defaults to `disabled`
280
281
  (`sent`/assisted input is rejected until the interventions ledger + a
281
- non-comparable marker exist). `runtimeAuth` (`openai-env`, the default, or `openai-egress`)
282
+ non-comparable marker exist). `runtimeAuth` (`openai-egress`, the default, or `openai-env`)
282
283
  declares the agent's runtime-auth channel, recorded as names only. On a live run, the engine
283
284
  resolves the registered terminal descriptor and requires
284
285
  `keyPlacement: in-sandbox-command-scoped` before creating a sandbox. With `openai-env`
@@ -300,10 +301,11 @@ instruction?, target?, entry?, host?, reasoningEffort?, stopWhen?, dwell? }` bec
300
301
  wall-clock kill; `maxUsd`/`maxJobs` are enforced fail-closed against the cost
301
302
  ledger after the session (a run whose known spend exceeds the cap fails closed,
302
303
  `HUMANISH_TERMINAL_CAPS_EXCEEDED`). Core records Codex provider spend as
303
- unpriced tokens, so without a cost probe no line can trip a positive `maxUsd`:
304
- a live run refuses `maxUsd > 0` before creating a sandbox
305
- (`HUMANISH_TERMINAL_UNPRICED_CAP`) unless the caller passes a `costProbe`
306
- hook. The sandbox's server-side timeout is the steps before the
304
+ unpriced tokens and measures no product, media or payment spend, so no line can trip a
305
+ positive `maxUsd`: a live run refuses `maxUsd > 0` before creating a sandbox
306
+ (`HUMANISH_TERMINAL_UNPRICED_CAP`). A terminal `maxUsd` cap has no adopter cost source
307
+ until [issue 347](https://github.com/danielgwilson/humanish/issues/347) lands;
308
+ `StudyDeps.costProbe` is a test seam. The sandbox's server-side timeout is the steps before the
307
309
  codex command (Node bootstrap, runtime version check, and product setup when
308
310
  `subject.product.install` is declared) plus `maxMinutes` plus a 5-minute
309
311
  teardown buffer; a live run refuses a `maxMinutes` that would take it past
@@ -1066,9 +1068,9 @@ contract.
1066
1068
  ## Terminal Cost Ledger And No-Spend Proof
1067
1069
 
1068
1070
  The terminal-product route (`src/routes/terminal/route.ts`), under the default
1069
- `runtimeAuth: openai-env`, passes a real provider key only to the in-sandbox agent
1070
- command; `openai-egress` gives the command a placeholder and keeps the key in an
1071
- external E2B header transform. Neither mode puts it in sandbox-global env or
1071
+ `runtimeAuth: openai-egress`, gives the in-sandbox agent command a placeholder and
1072
+ keeps the key in an external E2B header transform; `openai-env` passes a real
1073
+ provider key only to that command. Neither mode puts it in sandbox-global env or
1072
1074
  metadata, so the no-spend claim must be derived from a ledger, never asserted. The
1073
1075
  live run writes both to `terminal-ledgers.json` (a `cost` block + a
1074
1076
  `noSpendProof` block, additive to `humanish.terminal-ledgers.v1`).
@@ -1088,8 +1090,8 @@ discipline** that distinguishes three states and never conflates them:
1088
1090
  and is never coerced to `0`); `fullyMeasured` is true only when no line is null.
1089
1091
  Core meters only the `provider` line, populated from the actor trace's
1090
1092
  `tokenUsage.costUsd` when present (else `null`); `product`/`media`/`payment`
1091
- remain `null` unless an adapter supplies those signals through the shipped
1092
- cost-probe seam. The terminal route records the model as `codex` and does not pin Codex's
1093
+ remain `null`, because no adopter can supply those signals until
1094
+ [issue 347](https://github.com/danielgwilson/humanish/issues/347) lands. The terminal route records the model as `codex` and does not pin Codex's
1093
1095
  model, so a measured token count stays `usd: null` with `source: unpriced-token-usage`, and
1094
1096
  `knownTotalUsd: 0` with `fullyMeasured: false` means no line of this ledger was priced. E2B
1095
1097
  time for the terminal sandbox is not a line of this ledger, whose lines are checked against
@@ -1149,9 +1151,13 @@ finished | interrupted, mode, study?, pid, startedAt, updatedAt, completedAt?,
1149
1151
  signal?, outcome? }`. A record written by 0.108 or earlier also has `lab`, with the same value as
1150
1152
  `study`, and readers accept it.
1151
1153
  `outcome` carries the bundle's `verdict`, `participants`, `estimatedCostUsd` and
1152
- `estimatedCostComplete` when the run finishes, then the result's `ok` and `execution: { succeeded,
1153
- failures: [{ kind, message }], warnings? }` once the Observer has rendered. The
1154
- verdict is what the participants experienced; `execution` is whether the run
1154
+ `estimatedCostComplete` when the run finishes, with `ok` and `execution: { succeeded,
1155
+ failures: [{ kind, message }], warnings? }` copied from `run.json`'s `outcome`
1156
+ ([Run Outcome](run-bundle.md#run-outcome)) in the same publication. An Observer
1157
+ that does not render adds its `evidence` failure to both files. A record written
1158
+ before `run.json` carried an outcome got `ok` and `execution` after the Observer
1159
+ rendered and is the only copy of that run's `ok`; readers use it for such a run
1160
+ only. The verdict is what the participants experienced; `execution` is whether the run
1155
1161
  worked as an execution (`kind` is `harness`, `provider-cleanup`,
1156
1162
  `provider-policy`, `sandbox-cleanup`, `evidence`, `cap` or `run`), and `ok`
1157
1163
  reads both under the route's policy (`OUTCOME_POLICIES` in `src/run/judge.ts`).
@@ -1177,10 +1183,14 @@ and the estimate is a lower bound. A record without it, as every record before
1177
1183
 
1178
1184
  `interrupted` with `signal` (`SIGINT`, `SIGTERM` or `SIGHUP`) is written by
1179
1185
  the CLI's run command when that signal stops a live run outside post-run
1180
- analysis. It then kills the E2B sandboxes the run's `sandbox-receipts.ndjson`
1181
- names, writes `reclaim-receipt.json` as `humanish reclaim` does, and exits
1182
- 128+n. The bundle is not finished: run.json keeps its last live flush, and no
1183
- `outcome` is written. A `running` record that stopped refreshing still reads as
1186
+ analysis. It first prints one line saying it is stopping and refuses any further sandbox
1187
+ create. It then kills the E2B sandboxes the run's `sandbox-receipts.ndjson` names, each sandbox
1188
+ a create in flight reports as its id arrives, and each one E2B lists with the run's owner tags,
1189
+ writes `reclaim-receipt.json` as `humanish reclaim` does, rewrites each run file that names one of
1190
+ the run's raw sandbox ids with the marker and digest, as a finished run's publish does, and exits
1191
+ 128+n. The bundle is not finished: run.json keeps its last live flush, with
1192
+ `outcome: { state: interrupted, ok: false, signal, at }` when the run had
1193
+ flushed one, and status.json gets no `outcome`. A `running` record that stopped refreshing still reads as
1184
1194
  interrupted once stale. A library caller of `runStudy` gets no signal handling. A
1185
1195
  humanish older than this field rejects the state and falls back to the bundle's
1186
1196
  own liveness.
@@ -1192,8 +1202,10 @@ nothing at all until they completed.
1192
1202
 
1193
1203
  It is a derived index, not evidence. `run.json` remains the evidence-of-record;
1194
1204
  `verify` never gates on `status.json`, nothing in it is a claim about what a
1195
- participant did, and when the two disagree the bundle wins and the record is
1196
- rebuildable from it. A `running` record whose `updatedAt` is older than three
1205
+ participant did, and when the two disagree the bundle wins. Fields that cannot be
1206
+ rebuilt from `run.json`: `pid`, `startedAt`, `updatedAt` and `completedAt`, which
1207
+ no bundle records, and, on a record written before `run.json` carried an
1208
+ `outcome`, `outcome.ok` and `outcome.execution`. A `running` record whose `updatedAt` is older than three
1197
1209
  touch intervals is interrupted, not alive. A dropped connection or a killed
1198
1210
  terminal leaves exactly that shape, so the record reads as interrupted
1199
1211
  (`classifyRunStatus` is the one shared definition). Fields are
@@ -1210,16 +1222,20 @@ them is a claim about what a participant did.
1210
1222
  `humanish.run-index.v1` is the listing projection. One entry per run, read
1211
1223
  cheapest-source-first: the `status.json` record, else the bundle, else the run
1212
1224
  directory alone. `{ runId, derivedFrom: status | bundle | directory, liveness,
1213
- mode?, pid?, study?, startedAt?, updatedAt?, completedAt?, verdict?,
1214
- participants?, estimatedCostUsd?, estimatedCostComplete?, analysisCost?,
1215
- durationMs? }`. The point is cost: walking
1225
+ mode?, pid?, study?, startedAt?, updatedAt?, completedAt?, verdict?, ok?,
1226
+ failure?, participants?, estimatedCostUsd?, estimatedCostComplete?,
1227
+ analysisCost?, durationMs? }`. `ok` and `failure` (the first execution failure)
1228
+ are the run's own outcome; `runDisplay` (`src/run/display.ts`) reads them with
1229
+ `liveness` and `verdict`, so the TUI and `humanish stats` show a run with
1230
+ `ok: false` as failed whatever its verdict. The point is cost: walking
1216
1231
  every run tree and parsing every bundle measured 167ms on a 25-run project,
1217
1232
  against 16ms cold and 2.8ms warm here, which is what makes a surface that
1218
1233
  refreshes on a cadence affordable. `derivedFrom` is reported so a surprising row
1219
1234
  can be traced to the file it came from. A run with receipts and no outcome is
1220
1235
  `interrupted` (the shape a dropped connection leaves), and so is an
1221
1236
  in-progress bundle reached without a status record: there is no freshness to
1222
- judge, and nothing in it says the run finished.
1237
+ judge, and nothing in it says the run finished. A bundle with an `outcome` reached
1238
+ without a status record takes its liveness from `outcome.state`.
1223
1239
 
1224
1240
  `estimatedCostUsd` is the participants-and-desktops figure from the status
1225
1241
  record or bundle and excludes analysis. `estimatedCostComplete` says whether that
@@ -1378,7 +1394,7 @@ usage is known or bounded:
1378
1394
  - A request lost without a reply (it stalled, or the OpenAI provider's
1379
1395
  transport threw) is booked at its worst case before it is sent again: the
1380
1396
  latest reported request's input at the model's highest input rate plus
1381
- `actors[].maxOutputTokens` at its output rate, both from `src/run/pricing.ts`.
1397
+ `actor.maxOutputTokens` at its output rate, both from `src/run/pricing.ts`.
1382
1398
  The reply to the resent request replaces that input with its own. The
1383
1399
  request is sent again only when the estimate with that charge stays within
1384
1400
  `maxUsd` and the study budget. Otherwise the session ends `budget_reached`
@@ -1566,7 +1582,7 @@ of it fails.
1566
1582
  | `SubjectPhaseEvent` | the `subject-phase` `StudyEvent` through `onEvent`. Its `type` is `"subject-phase"`; the phase that was `type` is `name`, and `target` is `{ kind: "subject" }` or `{ kind: "participant", participant }`. `message`, `at`, `ok` and `durationMs` are unchanged |
1567
1583
  | `StudyOutcome.backend`, `LabBackend` | `outcome.route` |
1568
1584
  | `selectLabBackend`, the `routesTo*` predicates, `actorResolvesToTerminal` | `routeOf(config)` |
1569
- | `resolveStudyDryRun` | `RunStudyOptions.dryRun`; without it, `scenario.mode` decides |
1585
+ | `resolveStudyDryRun` | `RunStudyOptions.dryRun`; without it, `mode` decides |
1570
1586
  | `cuaLaneCount` | the `plan` `StudyEvent`, which lists every participant |
1571
1587
  | `resolveSeatUrl` | `parseStudy`, which refuses a participant entry that is not same-origin loopback; build the URL with `new URL(entry, serveUrl)` |
1572
1588
  | `cuaLaneValidationReason`, `sharedWorldValidationReason`, `concurrentSharedWorldValidationReason`, `externalPublicSharedWorldValidationReason` | `parseStudy`, which returns the same reason |
@@ -1682,7 +1698,9 @@ the stream shape, the meaningful-use rubric, and hard-failure rules.
1682
1698
  Review summarizes whether evidence supports the claim. It does not replace
1683
1699
  verification or maintainer acceptance.
1684
1700
 
1685
- `verdict` is the run gate result. `participants.reachedGoal` retains the count of
1701
+ `verdict` is what the participants experienced, collapsed to one word. Whether
1702
+ the run passed also reads `run.json`'s `outcome.ok` ([Run Outcome](run-bundle.md#run-outcome)):
1703
+ a run whose execution failed shows as failed beside a `pass` verdict. `participants.reachedGoal` retains the count of
1686
1704
  recorded successful sessions; it is not an independent adjudication of their
1687
1705
  claims. Computer-use review and Observer labels distinguish a participant's
1688
1706
  reported completion from a recorded `stopWhen` or dwell condition match. A
@@ -288,6 +288,51 @@ companion project validated records; generic HTTP serving of producer-owned raw
288
288
  analysis/correction JSON, execution receipts, atomic write temporaries and locks
289
289
  is disabled. Existing adapter captures and logs retain their contained routes.
290
290
 
291
+ ## Reading findings
292
+
293
+ `humanish review` prints a run's analysis findings after its participant review, and
294
+ `humanish analyze show` prints them alone. Each finding shows its title, impact, confidence and
295
+ recovery, the participants it affected out of those exposed, the frames its evidence cites with
296
+ their time since the first retained capture, the capture files, its next step and the latest
297
+ human review note. A live run's human output ends with up to three findings, one line each, and
298
+ the `review` command that prints all of them. `analyze show --json` prints the validated
299
+ analysis record and its corrections, as before. Reading findings never starts an analysis.
300
+
301
+ `review --json` carries the same view as its `analysis` field, schema
302
+ `humanish.analysis-findings.v1`, built in `src/cli/findings.ts`:
303
+
304
+ | Field | Meaning |
305
+ | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
306
+ | `schema` | `humanish.analysis-findings.v1`. |
307
+ | `runId` | The run the view reads. |
308
+ | `state` | `ready`, `stale`, `running`, `none`, `skipped`, `failed`, `dry_run` or `unavailable`. Only `ready` and `stale` carry findings. |
309
+ | `reason` | The stable code behind a state other than `ready`, or null: an automatic analysis reason such as `AUTOMATIC_ANALYSIS_KEY_MISSING`, a failed attempt's error code, or a load warning such as `ANALYSIS_SOURCE_CHANGED`. |
310
+ | `message` | What happened, in words. |
311
+ | `next` | The command that gets findings for this run, or null when no command can, as for a dry run or a run whose participants left no evidence. |
312
+ | `analysisId`, `status`, `provider`, `model`, `completedAt`, `estimatedCostUsd` | The analysis version `analyze show` selects: the newest ready one, else the newest stale one. Null without one. |
313
+ | `runPath`, `path` | The run directory and the version's `analysis.json`, relative to the project directory. |
314
+ | `summary`, `limitations` | The analysis's own summary and limitations. |
315
+ | `findings[]` | Highest priority first: `id`, `title`, `summary`, `impact` (`blocked_task`, `friction`, `recovery`, `uncertain`), `confidence` (`low`, `medium`, `high`), `recovery` (`recovered`, `not_observed`, `unknown`), `affected[]` (`streamId`, `label`), `exposedCount`, `evidence[]`, `nextStep` and `correction`. |
316
+ | `findings[].evidence[]` | Each cited item once: `id`, `streamId`, `kind`, `bases` (how the observations used it), `frame` (the latest retained capture's index from 0, null before any capture), `elapsedMs` (time since the first retained capture, not a video offset), `at` and `capture` (the file relative to the project directory, null for nonvisual evidence). |
317
+ | `findings[].correction` | The latest `analyze correct` note on that finding version (`status`, `reason`, `replacementClaim`), or null. |
318
+ | `warnings` | Load warnings for the selected version. |
319
+
320
+ The states without findings:
321
+
322
+ - `running`: the automatic analysis is queued or running. `next` is `review` again.
323
+ - `skipped`: the automatic analysis was skipped or refused, for example without
324
+ `OPENAI_API_KEY`. `next` is `analyze --max-cost 3`.
325
+ - `failed`: the latest attempt failed or was cancelled and no usable version exists. `next`
326
+ reruns it with the same analyst, `--provider codex` for the Codex account analyst.
327
+ - `none`: no analysis ran for this live run, for example with `review.analysis: false`.
328
+ - `dry_run`: a dry run has no participant evidence to analyze.
329
+ - `unavailable`: the run or its analysis records could not be read or checked, or the analysis
330
+ text matched a sensitive-text pattern (`ANALYSIS_SENSITIVE_TEXT_QUARANTINED`), which the
331
+ Observer withholds too.
332
+
333
+ `stale` lists the findings with a warning: the run's evidence changed after the analysis, and
334
+ `next` requests a new one. `analyze show` without `--json` exits 2 when the JSON form would.
335
+
291
336
  ## Human review and sharing
292
337
 
293
338
  ```bash
@@ -29,6 +29,14 @@ They do not suddenly include a separate analysis request. The bundle's
29
29
  `cost.estimatedTotalUsd`, `status.json`'s `outcome.estimatedCostUsd` and the run
30
30
  index's `estimatedCostUsd` stay participants and desktops only.
31
31
 
32
+ Each study's `passed` counts the runs that show as passed on every surface
33
+ (`runDisplay` in `src/run/display.ts`): a `pass` verdict and a run whose own `ok`
34
+ is not false. `passRate` divides it by `judged`, the runs with a verdict.
35
+ `totals.verdicts` counts the participants' verdicts; `totals.outcomes` counts
36
+ every selected run by its display state (`passed`, `failed`, `blocked`,
37
+ `timed_out`, `no_verdict`, `dry_run`, `interrupted`, `running`, `unknown`), and
38
+ the human output prints that line.
39
+
32
40
  ## One run's cost
33
41
 
34
42
  A run's analysis spend has one reader, `readAnalysisAccounting` in
@@ -18,6 +18,9 @@ the evidence. Everything else is a projection rebuilt from them:
18
18
  - `status.json` (`humanish.run-status.v1`) is a liveness index. When it disagrees with `run.json`,
19
19
  `run.json` wins. `verify` reads it, with the sandbox and reclaim receipts, only to say a run
20
20
  did not finish (`RUN_NOT_FINISHED`). That warning changes no check and no grade.
21
+ - A run's `ok` and execution outcome are in `run.json`'s `outcome`, and `status.json` copies them.
22
+ A `status.json` written before `run.json` carried an outcome holds the only copy of that run's
23
+ `ok`, so readers fall back to it for such a run only.
21
24
  - Analysis findings cite turns and artifacts in the bundle.
22
25
 
23
26
  ## Consequences
@@ -1,6 +1,7 @@
1
1
  # 0003: Managed paths bind to physical identities; cleanup uses create-time receipts
2
2
 
3
- Accepted in 0.15.1, extended in 0.36.0 (`humanish reclaim`).
3
+ Accepted in 0.15.1, extended in 0.36.0 (`humanish reclaim`) and after 0.110.0 (search by owner
4
+ tags).
4
5
 
5
6
  ## Context
6
7
 
@@ -18,11 +19,16 @@ Separately, an account-wide provider operation once destroyed unrelated infrastr
18
19
  - The process that creates a resource cleans it up with the handle it holds. After a crash,
19
20
  `humanish reclaim` kills sandboxes by the exact ids journaled at create time in the run's
20
21
  `sandbox-receipts.ndjson`, and reports each outcome.
21
- - No route lists an account's resources to find or verify its own.
22
+ - Nothing lists an account's resources unfiltered. Every sandbox carries its run's owner tags
23
+ (`tool`, `runId`, `runKey`), and reclaim lists only the sandboxes E2B matches to all three,
24
+ checks each tag again before it acts, and stops after three pages. A sandbox whose id never
25
+ reached a receipt, from a create that threw after E2B allocated or a process that died
26
+ mid-create, is otherwise unreachable until its timeout.
22
27
 
23
28
  ## Consequences
24
29
 
25
- - A shared operator key is safe to use: humanish reaches only resources it created.
30
+ - A shared operator key is safe to use: humanish reaches only resources it created. `runKey` keeps
31
+ two projects that reuse a run id from matching each other's sandboxes.
26
32
  - An exported bundle does not carry cleanup authority; `export` omits the receipt journal.
27
33
  - Unverified git metadata (arbitrary `gitdir:` redirects) is reported as unavailable instead of
28
34
  followed.
@@ -66,14 +66,17 @@ certify (see the conformance suite).
66
66
  6. **Claims match mechanism.** A config field that is parsed but not consumed warns; a
67
67
  document that overstates behavior is a defect; an evidence artifact never claims a
68
68
  stronger evidence class than its actor and scenario can support.
69
- 7. **Reclamation is by exact created id, never account-wide enumeration.** A route that
70
- creates a provider resource (an E2B sandbox, for example) tracks the exact id it created and
71
- reclaims and proves cleanup against that id alone (a returned found-and-killed boolean, a
72
- getInfo-by-id NotFound check). No route ever calls a provider's list-everything endpoint to
73
- discover or verify its own cleanup, because that endpoint enumerates the whole account/team,
74
- including resources humanish did not create. This is what makes a single shared operator key
75
- safe: humanish only ever reaches a resource it created, never anything else the key's owner
76
- is running.
69
+ 7. **Reclamation is by exact created id or the run's exact owner tags, never account-wide
70
+ enumeration.** A route that creates a provider resource (an E2B sandbox, for example) tracks
71
+ the exact id it created and reclaims and proves cleanup against that id alone (a returned
72
+ found-and-killed boolean, a getInfo-by-id NotFound check). No route ever calls a provider's
73
+ list endpoint unfiltered to discover or verify its own cleanup, because that enumerates the
74
+ whole account/team, including resources humanish did not create. The one listing is reclaim's
75
+ search for a sandbox whose id never reached the run: every sandbox carries its run's owner
76
+ tags (`tool`, `runId`, `runKey`), E2B filters by all three, reclaim checks each tag again
77
+ before it acts, and it reads at most three pages. This is what makes a single shared operator
78
+ key safe: humanish only ever reaches a resource it created, never anything else the key's
79
+ owner is running.
77
80
  The one narrow exception is an explicit, maintainer-run orphan sweep (never a default, never
78
81
  reachable by a normal run) that is itself opt-in gated (an explicit flag or env var) and
79
82
  documented as such at its call site.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "humanish",
3
- "version": "0.110.0",
3
+ "version": "0.110.2",
4
4
  "description": "Synthetic user research for apps, CLIs, and agent-facing product flows.",
5
5
  "keywords": [
6
6
  "cli",
@@ -56,10 +56,12 @@
56
56
  },
57
57
  "scripts": {
58
58
  "build": "tsc -p tsconfig.build.json && pnpm --filter humanish-observer build && pnpm --filter humanish-tui build && node scripts/finish-build.mjs",
59
- "check": "pnpm format:check && pnpm lint && pnpm knip && pnpm prose:check && pnpm vocabulary:check && pnpm site-css:check && pnpm typecheck && pnpm test && pnpm tui:test && pnpm build && pnpm guest-media:startup:proof && pnpm cli:startup:test && pnpm cli:preflight:test && pnpm cli:dotenv:test && pnpm tui:smoke",
59
+ "check": "pnpm format:check && pnpm lint && pnpm knip && pnpm prose:check && pnpm vocabulary:check && pnpm docs:check && pnpm site-css:check && pnpm typecheck && pnpm test && pnpm tui:test && pnpm build && pnpm guest-media:startup:proof && pnpm cli:startup:test && pnpm cli:preflight:test && pnpm cli:dotenv:test && pnpm cli:peer:test && pnpm cli:skill:test && pnpm tui:smoke",
60
60
  "cli:startup:test": "node scripts/terminal-startup-exit-proof.mjs",
61
61
  "cli:preflight:test": "node scripts/task-route-preflight-proof.mjs",
62
62
  "cli:dotenv:test": "node scripts/dotenv-proof.mjs",
63
+ "cli:peer:test": "node scripts/peer-install-proof.mjs",
64
+ "cli:skill:test": "node scripts/skill-first-run-proof.mjs",
63
65
  "humanish": "tsx src/cli.ts",
64
66
  "pack:dry-run": "npm pack --dry-run",
65
67
  "prepack": "pnpm build",
@@ -73,9 +75,10 @@
73
75
  "tui:smoke": "node scripts/tui-smoke.mjs",
74
76
  "tui:test": "pnpm --filter humanish-tui test",
75
77
  "release:dogfood": "node scripts/release-dogfood.mjs",
78
+ "bench": "tsx bench/run.ts",
76
79
  "browser-control:proof": "node scripts/browser-control-proof.mjs",
77
80
  "docs:generate": "tsx scripts/generate-cli-docs.ts",
78
- "docs:check": "tsx scripts/generate-cli-docs.ts --check && tsx scripts/check-doc-paths.ts && tsx scripts/check-doc-symbols.ts && tsx scripts/check-doc-comments.ts",
81
+ "docs:check": "tsx scripts/generate-cli-docs.ts --check && tsx scripts/check-doc-paths.ts && tsx scripts/check-doc-symbols.ts && tsx scripts/check-doc-comments.ts && tsx scripts/check-doc-study-fields.ts",
79
82
  "observer:browser:proof": "node scripts/observer-browser-proof.mjs",
80
83
  "observer:iframe:proof": "node --import tsx scripts/observer-iframe-proof.mjs",
81
84
  "observer:chrome:proof": "node scripts/observer-chrome-proof.mjs",
@@ -35,7 +35,7 @@ Everything it shows has a machine-readable equivalent, which is what you want:
35
35
  | browsing studies | `npx humanish study list --json` |
36
36
  | browsing runs | `npx humanish runs --json` |
37
37
  | starting a run | `npx humanish run <study> --json --no-open` |
38
- | a run's outcome | `npx humanish review --run <id> --json` |
38
+ | a run's findings | `npx humanish review --run <id> --json` |
39
39
  | communication setup | `npx humanish comms providers --json` and `npx humanish comms connections list --json` |
40
40
 
41
41
  If a human asks you to "open the TUI", tell them the command to type; do not run
@@ -289,7 +289,6 @@ Useful commands:
289
289
  ```bash
290
290
  npx humanish study list
291
291
  npx humanish study show first-run
292
- npx humanish watch first-run
293
292
  npx humanish run first-run --json --no-open
294
293
  ```
295
294
 
@@ -377,25 +376,30 @@ inbox page. See `docs/contracts/schemas.md` for the full `comms:` shape and
377
376
  ## First Proof Run
378
377
 
379
378
  Run the no-credentials path first. It is a dry run: humanish writes a synthetic
380
- run bundle and tests no target app behavior:
379
+ run bundle and tests no target app behavior. Every command in this block
380
+ finishes and exits, needs no key and spends nothing; lines marked
381
+ `# live only` need keys and spend money, so run them only when the user has
382
+ approved a live run. `run` prints the run id; use it in place of `<id>`.
381
383
 
382
384
  ```bash
383
- npx humanish doctor
384
- npx humanish watch
385
- npx humanish verify --run latest --json
386
- npx humanish feedback issue --run latest --repo example/app --format markdown
387
- ```
388
-
389
- For CI or non-interactive proof:
390
-
391
- ```bash
392
- npx humanish watch --json --no-open
385
+ npx humanish doctor --json
393
386
  npx humanish run first-run --json --no-open
387
+ npx humanish verify --run <id> --json
388
+ npx humanish review --run <id> --json
389
+ npx humanish observe --run <id>
390
+ npx humanish run try-live --json --no-open # live only
391
+ npx humanish feedback issue --run <id> --repo example/app --format markdown # live only
394
392
  ```
395
393
 
396
- The feedback command prints a public-safe Markdown draft. It must not call the
397
- GitHub API, require a token, update Projects, use provider credits, or claim
398
- product behavior proof from a dry run.
394
+ `observe` prints the path of the run's Observer page for the person you are
395
+ working for. Without an interactive terminal, `observe` and `watch` print the
396
+ result and exit; at a person's terminal they serve the Observer until Ctrl-C.
397
+
398
+ `feedback issue` drafts from a live run only and refuses a dry run with
399
+ `HUMANISH_FEEDBACK_REQUIRES_LIVE_RUN`. Its Markdown draft is checked for
400
+ secret, key, token and local-path shapes; it is not checked for personal data. It
401
+ must not call the GitHub API, require a token, update Projects, use provider
402
+ credits, or claim product behavior proof from a dry run.
399
403
 
400
404
  When the target app can run locally, prove a known browser path with a
401
405
  scripted-browser study after starting the app on loopback. The study replays a
@@ -414,8 +418,8 @@ no browser.
414
418
  # in another terminal, start the target app on 127.0.0.1 or localhost
415
419
  npx humanish study show <study> --json
416
420
  npx humanish run <study> --json --no-open
417
- npx humanish verify --run latest --json
418
- npx humanish observe --run latest --open # for a person; it serves until Ctrl-C
421
+ npx humanish verify --run <id> --json
422
+ npx humanish observe --run <id> --open # for a person; at their terminal it serves until Ctrl-C
419
423
  ```
420
424
 
421
425
  Do not use `humanish watch --count ...` as a substitute for a scripted-browser
@@ -442,7 +446,7 @@ Do not paste values into files, prompts, run bundles, issue drafts, or logs.
442
446
  Load local values only at invocation time:
443
447
 
444
448
  ```bash
445
- npx humanish watch .humanish/studies/local-live.yaml --dotenv .humanish/local/provider.env
449
+ npx humanish run .humanish/studies/local-live.yaml --dotenv .humanish/local/provider.env --json --no-open
446
450
  ```
447
451
 
448
452
  When choosing dogfood targets, prefer apps, CLIs, or agent-facing tools with a
@@ -454,7 +458,21 @@ publish their names, screenshots, logs, source snippets, or operational details.
454
458
 
455
459
  ## Reporting Back
456
460
 
457
- Report:
461
+ After a live run, read its findings before anything else:
462
+
463
+ ```bash
464
+ npx humanish review --run <id> # the verdict, then each analysis finding
465
+ npx humanish review --run <id> --json # the same; findings are under `analysis`
466
+ ```
467
+
468
+ Report each finding to the user: its title, impact, confidence and recovery, the participants it
469
+ affected, and the capture files it cites. These come from the post-run analysis of the evidence.
470
+ Label the participant's own report in the review summary as the participant's account. When
471
+ `analysis.state` is not `ready`, report `analysis.message` and the `analysis.next` command in
472
+ place of findings; `analysis.next` is null when no command can produce them. `latest` moves with
473
+ every run, including dry runs, so use the run id the run printed.
474
+
475
+ Then report:
458
476
 
459
477
  - files changed in the target repo;
460
478
  - exact proof commands run;