humanish 0.115.0 → 0.117.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (195) hide show
  1. package/CHANGELOG.md +41 -0
  2. package/README.md +7 -6
  3. package/TELEMETRY.md +6 -0
  4. package/dist/actors/codex/app-server-ui.js +11 -4
  5. package/dist/actors/codex/restricted-participant-policy.d.ts +0 -2
  6. package/dist/actors/codex/restricted-participant-policy.js +4 -32
  7. package/dist/actors/codex/restricted-participant.d.ts +5 -0
  8. package/dist/actors/codex/restricted-participant.js +3 -2
  9. package/dist/actors/computer-use/actor.d.ts +4 -0
  10. package/dist/actors/computer-use/actor.js +2 -0
  11. package/dist/actors/computer-use/loop/actions.d.ts +4 -4
  12. package/dist/actors/computer-use/loop/actions.js +25 -18
  13. package/dist/actors/computer-use/loop/debrief.js +61 -27
  14. package/dist/actors/computer-use/loop/session.d.ts +3 -0
  15. package/dist/actors/computer-use/loop/session.js +9 -1
  16. package/dist/actors/computer-use/loop/trace.js +1 -0
  17. package/dist/actors/computer-use/loop/types.d.ts +16 -14
  18. package/dist/actors/computer-use/loop.d.ts +1 -1
  19. package/dist/actors/computer-use/loop.js +1 -1
  20. package/dist/actors/computer-use/wait.d.ts +74 -0
  21. package/dist/actors/computer-use/wait.js +100 -0
  22. package/dist/actors/contract.d.ts +14 -0
  23. package/dist/actors/scripted-browser/steps.js +6 -5
  24. package/dist/analysis/analysis-limits.d.ts +35 -0
  25. package/dist/analysis/analysis-limits.js +41 -0
  26. package/dist/analysis/validation.js +14 -21
  27. package/dist/browser-control/protocol.d.ts +86 -6
  28. package/dist/browser-control/protocol.js +13 -6
  29. package/dist/cli/commands/analyze.js +2 -1
  30. package/dist/cli/commands/notes.js +2 -1
  31. package/dist/cli/commands/run-signals.js +5 -1
  32. package/dist/cli/commands/runs.js +2 -1
  33. package/dist/cli/commands/setup.js +3 -3
  34. package/dist/cli/commands/study-format.d.ts +5 -2
  35. package/dist/cli/commands/study-format.js +29 -9
  36. package/dist/cli/commands/study-route-computer-use.js +2 -2
  37. package/dist/cli/commands/study-route-scripted.js +1 -1
  38. package/dist/cli/commands/tui.js +59 -17
  39. package/dist/cli/doctor.js +9 -2
  40. package/dist/cli/findings-text.d.ts +5 -0
  41. package/dist/cli/findings-text.js +169 -0
  42. package/dist/cli/findings.d.ts +0 -4
  43. package/dist/cli/findings.js +5 -167
  44. package/dist/cli/io.js +6 -1
  45. package/dist/cli/update-check.js +4 -1
  46. package/dist/comms/connections.js +4 -3
  47. package/dist/comms/setup.js +18 -12
  48. package/dist/evidence/redaction.js +9 -1
  49. package/dist/evidence/secret-scrub.d.ts +15 -5
  50. package/dist/evidence/secret-scrub.js +67 -9
  51. package/dist/feedback/draft.js +1 -1
  52. package/dist/feedback/export-bundle.js +18 -14
  53. package/dist/feedback/feedback.js +25 -7
  54. package/dist/guest/desktop-executor.js +3 -1
  55. package/dist/guest/desktop-keys.js +59 -5
  56. package/dist/keys/key-resolution.d.ts +9 -0
  57. package/dist/keys/key-resolution.js +60 -48
  58. package/dist/keys/key-status.d.ts +2 -0
  59. package/dist/keys/key-status.js +9 -16
  60. package/dist/observer/pinned-files.js +1 -1
  61. package/dist/observer/render.js +3 -2
  62. package/dist/observer-app.html +5 -5
  63. package/dist/routes/computer-use/bundle-parts.d.ts +4 -4
  64. package/dist/routes/computer-use/bundle-parts.js +2 -2
  65. package/dist/routes/computer-use/bundle.d.ts +4 -5
  66. package/dist/routes/computer-use/bundle.js +1 -3
  67. package/dist/routes/computer-use/e2b-desktop/prepare.js +40 -31
  68. package/dist/routes/computer-use/fanout-records.js +10 -11
  69. package/dist/routes/computer-use/live-phase.d.ts +1 -1
  70. package/dist/routes/computer-use/live-phase.js +0 -1
  71. package/dist/routes/computer-use/local-vm.js +5 -2
  72. package/dist/routes/computer-use/participant-execution.d.ts +3 -2
  73. package/dist/routes/computer-use/participant-model.js +9 -0
  74. package/dist/routes/computer-use/participant-runs.d.ts +6 -2
  75. package/dist/routes/computer-use/participant-runs.js +9 -5
  76. package/dist/routes/computer-use/plan.d.ts +3 -0
  77. package/dist/routes/computer-use/plan.js +18 -13
  78. package/dist/routes/computer-use/rerun-selection.js +2 -2
  79. package/dist/routes/computer-use/result.js +3 -6
  80. package/dist/routes/computer-use/setup.d.ts +4 -6
  81. package/dist/routes/computer-use/setup.js +2 -6
  82. package/dist/routes/computer-use/single-bundle.d.ts +4 -5
  83. package/dist/routes/computer-use/single-bundle.js +22 -25
  84. package/dist/routes/computer-use/subject-projection.d.ts +4 -8
  85. package/dist/routes/computer-use/subject-projection.js +9 -31
  86. package/dist/routes/computer-use/types.d.ts +5 -42
  87. package/dist/routes/scripted/scenario.js +28 -12
  88. package/dist/routes/scripted/session-result.js +6 -3
  89. package/dist/routes/shared-world/bundle-records.js +2 -8
  90. package/dist/routes/shared-world/participant-specs.d.ts +0 -2
  91. package/dist/routes/shared-world/participant-specs.js +1 -20
  92. package/dist/routes/shared-world/plan.d.ts +3 -0
  93. package/dist/routes/shared-world/plan.js +25 -4
  94. package/dist/routes/shared-world/setup.js +2 -2
  95. package/dist/routes/terminal/ledger.d.ts +9 -9
  96. package/dist/routes/terminal/ledger.js +63 -40
  97. package/dist/routes/terminal/live-finish.js +7 -10
  98. package/dist/routes/terminal/live-sandbox.d.ts +1 -1
  99. package/dist/routes/terminal/live-sandbox.js +14 -11
  100. package/dist/routes/terminal/plan.d.ts +3 -0
  101. package/dist/routes/terminal/plan.js +8 -5
  102. package/dist/routes/terminal/sandbox.d.ts +4 -3
  103. package/dist/routes/terminal/sandbox.js +5 -5
  104. package/dist/routes/terminal/token-usage.d.ts +1 -2
  105. package/dist/routes/terminal/token-usage.js +4 -6
  106. package/dist/routes/terminal/types.d.ts +9 -1
  107. package/dist/run/bundle-shape.js +6 -1
  108. package/dist/run/bundle.d.ts +5 -3
  109. package/dist/run/contained-output.d.ts +53 -5
  110. package/dist/run/contained-output.js +141 -42
  111. package/dist/run/detail.d.ts +2 -1
  112. package/dist/run/detail.js +12 -7
  113. package/dist/run/escape-sequences.d.ts +11 -0
  114. package/dist/run/escape-sequences.js +93 -0
  115. package/dist/run/evidence-files.d.ts +8 -8
  116. package/dist/run/evidence-files.js +19 -23
  117. package/dist/run/locate.d.ts +32 -3
  118. package/dist/run/locate.js +45 -40
  119. package/dist/run/notes.d.ts +0 -8
  120. package/dist/run/notes.js +5 -51
  121. package/dist/run/outcomes.d.ts +24 -0
  122. package/dist/run/outcomes.js +19 -0
  123. package/dist/run/paths.d.ts +2 -0
  124. package/dist/run/paths.js +5 -1
  125. package/dist/run/preflight-receipts.js +13 -8
  126. package/dist/run/project.js +7 -4
  127. package/dist/run/reclaim-outcomes.d.ts +8 -6
  128. package/dist/run/reclaim-outcomes.js +10 -6
  129. package/dist/run/reclaim.d.ts +5 -0
  130. package/dist/run/reclaim.js +43 -16
  131. package/dist/run/run-clock.d.ts +50 -0
  132. package/dist/run/run-clock.js +66 -0
  133. package/dist/run/run.js +5 -2
  134. package/dist/run/sandbox-ids.d.ts +7 -2
  135. package/dist/run/sandbox-ids.js +27 -11
  136. package/dist/run/secrets.d.ts +2 -1
  137. package/dist/run/secrets.js +18 -22
  138. package/dist/run/stored-runs.js +14 -7
  139. package/dist/run/transient-comms-secrets.js +6 -17
  140. package/dist/run-study.js +22 -13
  141. package/dist/study/adapter-scorer-loader.js +6 -4
  142. package/dist/study/config.js +4 -1
  143. package/dist/study/discover.js +9 -6
  144. package/dist/study/init-paths.js +7 -5
  145. package/dist/study/keys.d.ts +2 -0
  146. package/dist/study/keys.js +2 -0
  147. package/dist/study/migrate/v2.js +2 -1
  148. package/dist/study/parse/actors.js +8 -0
  149. package/dist/study/persona-resolve.d.ts +2 -2
  150. package/dist/study/persona-resolve.js +9 -5
  151. package/dist/study/plan-participants.d.ts +4 -0
  152. package/dist/study/plan-participants.js +4 -0
  153. package/dist/study/plan-types.d.ts +2 -2
  154. package/dist/study/plan.d.ts +5 -3
  155. package/dist/study/plan.js +18 -9
  156. package/dist/study/preflight-probes.js +11 -3
  157. package/dist/study/preflight.js +9 -5
  158. package/dist/study/summary.d.ts +8 -2
  159. package/dist/study/summary.js +20 -22
  160. package/dist/study/types.d.ts +14 -0
  161. package/dist/study/validation.d.ts +5 -0
  162. package/dist/study/validation.js +8 -0
  163. package/dist/study/warnings.d.ts +19 -12
  164. package/dist/study/warnings.js +50 -17
  165. package/dist/subject/serve.d.ts +3 -0
  166. package/dist/subject/serve.js +4 -1
  167. package/dist/substrates/detached.js +14 -7
  168. package/dist/substrates/e2b/connection.d.ts +12 -0
  169. package/dist/substrates/e2b/connection.js +35 -0
  170. package/dist/substrates/e2b/desktop-executor.d.ts +1 -5
  171. package/dist/substrates/e2b/desktop-executor.js +45 -12
  172. package/dist/substrates/e2b/lifetime.d.ts +24 -4
  173. package/dist/substrates/e2b/lifetime.js +41 -4
  174. package/dist/substrates/e2b/sandbox.d.ts +4 -3
  175. package/dist/substrates/e2b/sandbox.js +13 -18
  176. package/dist/substrates/e2b/sdk.d.ts +20 -9
  177. package/dist/substrates/local/capacity.d.ts +18 -2
  178. package/dist/substrates/local/capacity.js +21 -2
  179. package/dist/substrates/local/runtime-host.js +21 -14
  180. package/dist/substrates/local/runtime.js +3 -11
  181. package/dist/tui/contract.d.ts +30 -5
  182. package/dist/tui/contract.js +4 -1
  183. package/dist/tui-app.js +150 -147
  184. package/dist/verify/artifacts.d.ts +9 -2
  185. package/dist/verify/artifacts.js +37 -13
  186. package/dist/verify/liveness.js +19 -5
  187. package/dist/verify/verify.js +54 -27
  188. package/docs/architecture/browser-control.md +10 -4
  189. package/docs/architecture/guest-desktop.md +7 -2
  190. package/docs/architecture/state-driven-executor.md +4 -2
  191. package/docs/contracts/run-bundle.md +18 -10
  192. package/docs/contracts/schemas.md +46 -29
  193. package/docs/contracts/study-analysis.md +10 -1
  194. package/docs/product/automatic-analysis.md +7 -1
  195. package/package.json +2 -2
@@ -2,6 +2,11 @@ import { type PreparedRunArtifactPaths } from "../run/paths.js";
2
2
  import type { RunBundle } from "../run/bundle.js";
3
3
  /** Public-safety and evidence-reference findings stop at this many per list. */
4
4
  export declare const MAX_REPORTED_FINDINGS = 50;
5
+ /**
6
+ * The most entries, files and directories, the public-safety scan visits in one run. Bundle export
7
+ * refuses a run with more, so the scan never passes a run whose files it did not all read.
8
+ */
9
+ export declare const MAX_RUN_ENTRIES = 10000;
5
10
  export declare function missingLocalEvidenceArtifacts(runPaths: PreparedRunArtifactPaths, bundle: RunBundle): Promise<string[]>;
6
11
  /**
7
12
  * The PNG frames that actor traces register as stream screenshots, relative to the run root.
@@ -44,8 +49,10 @@ interface RegisteredStreamMedia {
44
49
  }
45
50
  /**
46
51
  * Scans every run file for secret and path patterns and returns the findings. A file that is not
47
- * registered stream media and that the scan cannot read as text (readPlainText), or cannot read at
48
- * all, goes to `unscanned`, so the caller can keep the run from grading share_ready.
52
+ * registered stream media and that the scan cannot read as text (readPlainText), cannot read at
53
+ * all, or that holds more than RUN_ARTIFACT_MAX_BYTES goes to `unscanned`, so the caller can keep
54
+ * the run from grading share_ready. So does a directory it cannot list, and the rest of the run
55
+ * after MAX_RUN_ENTRIES entries.
49
56
  */
50
57
  export declare function scanRunPublicSafetyArtifacts(runPaths: PreparedRunArtifactPaths, derivedFindings: string[], media: RegisteredStreamMedia, unscanned: string[]): Promise<string[]>;
51
58
  export {};
@@ -6,15 +6,20 @@ import { scanEncodedTextCached } from "../evidence/encoded-text.js";
6
6
  import { readPlainText } from "../evidence/plain-text.js";
7
7
  import { containsSensitive } from "../evidence/redaction.js";
8
8
  import { isLocalEvidenceArtifactPath, isRiskyPublicArtifactPath, validatePreparedRunArtifactPaths, } from "../run/paths.js";
9
- import { openContainedRegularFile } from "../run/contained-output.js";
9
+ import { openContainedRegularFile, RUN_ARTIFACT_MAX_BYTES } from "../run/contained-output.js";
10
10
  import { holdsKeyedSandboxId } from "../run/sandbox-ids.js";
11
11
  import { SANDBOX_RECEIPTS_ARTIFACT } from "../run/sandbox-receipts.js";
12
- import { readSafeRunArtifactBytes, readSafeRunArtifactJson } from "../run/locate.js";
12
+ import { readSafeRunArtifact, readSafeRunArtifactBytes, readSafeRunArtifactJson, } from "../run/locate.js";
13
13
  import { isRecord } from "../run/type-guards.js";
14
14
  import { TERMINAL_EVENTS_ARTIFACT } from "../run/terminal-contract.js";
15
15
  import { isZeroEventTerminalTrace } from "./actor.js";
16
16
  /** Public-safety and evidence-reference findings stop at this many per list. */
17
17
  export const MAX_REPORTED_FINDINGS = 50;
18
+ /**
19
+ * The most entries, files and directories, the public-safety scan visits in one run. Bundle export
20
+ * refuses a run with more, so the scan never passes a run whose files it did not all read.
21
+ */
22
+ export const MAX_RUN_ENTRIES = 10_000;
18
23
  export async function missingLocalEvidenceArtifacts(runPaths, bundle) {
19
24
  const recordings = new Map(bundle.streams.flatMap((stream) => stream.recording ? [[stream.recording.path, stream.recording]] : []));
20
25
  const requiredPaths = new Map();
@@ -97,13 +102,18 @@ export async function missingLocalEvidenceArtifacts(runPaths, bundle) {
97
102
  }
98
103
  continue;
99
104
  }
100
- const bytes = await readSafeRunArtifactBytes(runPaths, artifactPath);
101
- if (!bytes || (bytes.length === 0 && !requirements.allowEmpty)) {
105
+ const artifact = await readSafeRunArtifact(runPaths, artifactPath);
106
+ if (artifact.state === "limit") {
107
+ missing.push(`${artifactPath} (larger than the ${RUN_ARTIFACT_MAX_BYTES} bytes verify reads)`);
108
+ continue;
109
+ }
110
+ if (artifact.state === "unavailable" ||
111
+ (artifact.bytes.length === 0 && !requirements.allowEmpty)) {
102
112
  missing.push(artifactPath);
103
113
  continue;
104
114
  }
105
115
  if (requirements.screenshot) {
106
- const imageError = screenshotEvidenceError(artifactPath, bytes);
116
+ const imageError = screenshotEvidenceError(artifactPath, artifact.bytes);
107
117
  if (imageError) {
108
118
  missing.push(`${artifactPath} (${imageError})`);
109
119
  }
@@ -280,17 +290,21 @@ export function rawScreenshotStreamIds(bundle, redactedShapeFrames) {
280
290
  }
281
291
  /**
282
292
  * Scans every run file for secret and path patterns and returns the findings. A file that is not
283
- * registered stream media and that the scan cannot read as text (readPlainText), or cannot read at
284
- * all, goes to `unscanned`, so the caller can keep the run from grading share_ready.
293
+ * registered stream media and that the scan cannot read as text (readPlainText), cannot read at
294
+ * all, or that holds more than RUN_ARTIFACT_MAX_BYTES goes to `unscanned`, so the caller can keep
295
+ * the run from grading share_ready. So does a directory it cannot list, and the rest of the run
296
+ * after MAX_RUN_ENTRIES entries.
285
297
  */
286
298
  export async function scanRunPublicSafetyArtifacts(runPaths, derivedFindings, media, unscanned) {
287
299
  const findings = [];
288
300
  await validatePreparedRunArtifactPaths(runPaths);
289
- await scanRunPublicSafetyDirectory(runPaths, "", findings, derivedFindings, media, unscanned);
301
+ await scanRunPublicSafetyDirectory(runPaths, "", findings, derivedFindings, media, unscanned, {
302
+ entries: 0,
303
+ });
290
304
  await validatePreparedRunArtifactPaths(runPaths);
291
305
  return findings;
292
306
  }
293
- async function scanRunPublicSafetyDirectory(runPaths, relativeDirectory, findings, derivedFindings, media, unscanned) {
307
+ async function scanRunPublicSafetyDirectory(runPaths, relativeDirectory, findings, derivedFindings, media, unscanned, walk) {
294
308
  // Each authority has its own finding budget. Derived files must never consume
295
309
  // the source scan's budget and make an unscanned recording appear verified.
296
310
  if (findings.length >= MAX_REPORTED_FINDINGS && derivedFindings.length >= MAX_REPORTED_FINDINGS) {
@@ -299,8 +313,18 @@ async function scanRunPublicSafetyDirectory(runPaths, relativeDirectory, finding
299
313
  const current = relativeDirectory
300
314
  ? path.join(runPaths.physicalRunRoot, ...relativeDirectory.split("/"))
301
315
  : runPaths.physicalRunRoot;
302
- const entries = await readdir(current).catch(() => []);
316
+ const entries = await readdir(current).catch(() => null);
317
+ if (entries === null) {
318
+ unscanned.push(`${relativeDirectory || "."}/ (could not be listed)`);
319
+ return;
320
+ }
303
321
  for (const entryName of entries) {
322
+ walk.entries += 1;
323
+ if (walk.entries > MAX_RUN_ENTRIES) {
324
+ if (walk.entries === MAX_RUN_ENTRIES + 1)
325
+ unscanned.push(`${relativeDirectory || "."}/ (the scan stops after ${MAX_RUN_ENTRIES} entries)`);
326
+ return;
327
+ }
304
328
  const relativePath = relativeDirectory ? `${relativeDirectory}/${entryName}` : entryName;
305
329
  const stats = await lstat(path.join(current, entryName), { bigint: true }).catch(() => null);
306
330
  const selectedFindings = !stats?.isDirectory() &&
@@ -328,7 +352,7 @@ async function scanRunPublicSafetyDirectory(runPaths, relativeDirectory, finding
328
352
  if (stats.isDirectory()) {
329
353
  // A directory named analysis.json is not an owned record. Its children
330
354
  // can contain source evidence even after derived findings are saturated.
331
- await scanRunPublicSafetyDirectory(runPaths, relativePath, findings, derivedFindings, media, unscanned);
355
+ await scanRunPublicSafetyDirectory(runPaths, relativePath, findings, derivedFindings, media, unscanned, walk);
332
356
  continue;
333
357
  }
334
358
  if (selectedFindings.length >= MAX_REPORTED_FINDINGS)
@@ -342,8 +366,8 @@ async function scanRunPublicSafetyDirectory(runPaths, relativeDirectory, finding
342
366
  if (media.recordingPaths.has(relativePath) || media.screenshotPaths.has(relativePath)) {
343
367
  continue;
344
368
  }
345
- // The scan reads a file by its bytes, not its name. A file that is not text, or that could
346
- // not be read at all, holds bytes the scan never saw.
369
+ // The scan reads a file by its bytes, not its name. A file that is not text, that could not
370
+ // be read, or that is over the read limit holds bytes the scan never saw.
347
371
  const bytes = await readSafeRunArtifactBytes(runPaths, relativePath).catch(() => null);
348
372
  const decoded = bytes === null ? undefined : readPlainText(bytes);
349
373
  if (decoded === undefined || !decoded.ok) {
@@ -4,7 +4,8 @@
4
4
  // `ok: true` is not read as "the run finished" or "its sandboxes stopped". It never flips `ok`.
5
5
  import { sandboxIdDigest } from "../evidence/redaction.js";
6
6
  import path from "node:path";
7
- import { readRunJsonIfExists, readRunTextIfExists } from "../run/locate.js";
7
+ import { refusalText } from "../run/contained-output.js";
8
+ import { readRunJsonIfExists, readRunTextIfExists, runJsonValue } from "../run/locate.js";
8
9
  import { isRecord } from "../run/type-guards.js";
9
10
  import { RECLAIM_RECEIPT_ARTIFACT } from "../run/reclaim.js";
10
11
  import { runLiveness } from "../run/liveness.js";
@@ -22,7 +23,8 @@ const GONE_STATES = new Set(["killed", "already-gone"]);
22
23
  */
23
24
  export async function runNotFinished(runPaths, bundle, nowMs = Date.now()) {
24
25
  const runId = path.basename(runPaths.absoluteRunRoot);
25
- const status = await readRunJsonIfExists(runPaths, RUN_STATUS_FILE);
26
+ // A status.json it refuses is no usable status, as a missing one is: the bundle decides.
27
+ const status = runJsonValue(await readRunJsonIfExists(runPaths, RUN_STATUS_FILE));
26
28
  const { liveness, record } = runLiveness(runId, status, bundle, nowMs);
27
29
  if (liveness === "finished")
28
30
  return { warnings: [] };
@@ -56,11 +58,23 @@ export async function runNotFinished(runPaths, bundle, nowMs = Date.now()) {
56
58
  /** What the reclaim receipt says about this run's sandboxes, and what to run. */
57
59
  async function reclaimState(runPaths, runId) {
58
60
  const journal = await readRunTextIfExists(runPaths, SANDBOX_RECEIPTS_ARTIFACT);
61
+ const command = `\`${cli(`reclaim --run ${runId}`)}\``;
62
+ // A journal it cannot read hides which sandboxes the run made, so no receipt can cover them.
63
+ if (journal.status === "refused")
64
+ return {
65
+ sandboxes: "unconfirmed",
66
+ text: `Sandboxes unconfirmed: ${refusalText(SANDBOX_RECEIPTS_ARTIFACT, journal)}, so the sandboxes it journaled are not known; check the file, then run ${command}.`,
67
+ };
59
68
  // Matched by digest: a reclaim receipt names each sandbox by digest, and one written before it
60
69
  // did holds the raw id, which is digested here.
61
- const journaled = new Set((journal === null ? [] : parseSandboxReceipts(journal)).map((receipt) => sandboxIdDigest(receipt.sandboxId)));
62
- const command = `\`${cli(`reclaim --run ${runId}`)}\``;
63
- const receipt = reclaimReceipt(await readRunJsonIfExists(runPaths, RECLAIM_RECEIPT_ARTIFACT), runId);
70
+ const journaled = new Set((journal.status === "missing" ? [] : parseSandboxReceipts(journal.text)).map((receipt) => sandboxIdDigest(receipt.sandboxId)));
71
+ const receiptRead = await readRunJsonIfExists(runPaths, RECLAIM_RECEIPT_ARTIFACT);
72
+ if (receiptRead.status === "refused")
73
+ return {
74
+ sandboxes: "unknown",
75
+ text: `Sandboxes unknown: ${refusalText(RECLAIM_RECEIPT_ARTIFACT, receiptRead)}, and the run journaled ${plural(journaled.size, "sandbox", "sandboxes")}; check the file, then run ${command}.`,
76
+ };
77
+ const receipt = reclaimReceipt(runJsonValue(receiptRead), runId);
64
78
  if (receipt === undefined)
65
79
  return {
66
80
  sandboxes: "unknown",
@@ -6,8 +6,10 @@ import { containsSensitive, REDACTED_SANDBOX_ID } from "../evidence/redaction.js
6
6
  import { keyedSandboxIds, readRunSandboxIds } from "../run/sandbox-ids.js";
7
7
  import { validatePreparedRunArtifactPaths } from "../run/paths.js";
8
8
  import { RUN_BUNDLE_FILE, RUN_BUNDLE_SCHEMA } from "../run/bundle.js";
9
+ import { ContainedReadRefusedError, refusalText } from "../run/contained-output.js";
9
10
  import { isCleanupResult, isRunBundle } from "../run/bundle-shape.js";
10
- import { readRunJsonIfExists, readRunTextIfExists, resolveRunPath } from "../run/locate.js";
11
+ import { readRunJsonIfExists, readRunTextIfExists, resolveRunPath, runJsonValue, } from "../run/locate.js";
12
+ import { SANDBOX_RECEIPTS_ARTIFACT } from "../run/sandbox-receipts.js";
11
13
  import { isRecord } from "../run/type-guards.js";
12
14
  import { runNotFoundMessage } from "../run/run-not-found.js";
13
15
  import { actorVerdictConsistencyFindings, noEngagementActorFindings, validateCodexAppServerEvidence, validateTerminalProductEvidence, } from "./actor.js";
@@ -75,26 +77,34 @@ export async function verifyResolvedRun(cwd, runInput, runPaths) {
75
77
  };
76
78
  }
77
79
  const bundlePath = path.join(runPaths.absoluteRunRoot, RUN_BUNDLE_FILE);
78
- const bundle = await readRunJsonIfExists(runPaths, RUN_BUNDLE_FILE);
79
- const cleanupJson = await readRunJsonIfExists(runPaths, "cleanup.json");
80
- const reviewJson = await readRunJsonIfExists(runPaths, "review.json");
81
- const reviewMarkdown = await readRunTextIfExists(runPaths, "review.md");
82
- checks.push(...bundlePresenceChecks(bundle, {
83
- json: reviewJson !== null,
84
- markdown: reviewMarkdown !== null,
85
- }));
80
+ const bundleRead = await readRunJsonIfExists(runPaths, RUN_BUNDLE_FILE);
81
+ const bundle = runJsonValue(bundleRead);
82
+ const cleanupRead = await readRunJsonIfExists(runPaths, "cleanup.json");
83
+ const review = {
84
+ json: await readRunJsonIfExists(runPaths, "review.json"),
85
+ markdown: await readRunTextIfExists(runPaths, "review.md"),
86
+ };
87
+ checks.push(...bundlePresenceChecks(bundleRead, review));
86
88
  const derivedPublicSafetyFindings = [];
87
89
  const unscannedArtifacts = [];
88
90
  const sandboxIdFiles = [];
91
+ let journaledSandboxIds = [];
92
+ try {
93
+ journaledSandboxIds = await readRunSandboxIds(runPaths);
94
+ }
95
+ catch (error) {
96
+ if (!(error instanceof ContainedReadRefusedError))
97
+ throw error;
98
+ // Without the journal's ids, verify cannot clear any file of naming one.
99
+ unscannedArtifacts.push(SANDBOX_RECEIPTS_ARTIFACT);
100
+ }
89
101
  const publicSafetyFindings = await scanRunPublicSafetyArtifacts(runPaths, derivedPublicSafetyFindings, {
90
102
  recordingPaths: new Set(isRunBundle(bundle)
91
103
  ? bundle.streams.flatMap((stream) => (stream.recording ? [stream.recording.path] : []))
92
104
  : []),
93
105
  screenshotPaths: isRunBundle(bundle) ? streamScreenshotPaths(bundle) : new Set(),
94
106
  // A run from before 0.110 with no receipts still names its ids in run.json.
95
- sandboxIds: [
96
- ...new Set([...(await readRunSandboxIds(runPaths)), ...keyedSandboxIds(bundle)]),
97
- ],
107
+ sandboxIds: [...new Set([...journaledSandboxIds, ...keyedSandboxIds(bundle)])],
98
108
  sandboxIdFiles,
99
109
  }, unscannedArtifacts);
100
110
  // JSON escapes can hide a sensitive value from the byte scan while the
@@ -129,7 +139,7 @@ export async function verifyResolvedRun(cwd, runInput, runPaths) {
129
139
  ? `invalid evidence artifact references: ${invalidEvidenceReferences.join(", ")}`
130
140
  : `missing local evidence artifacts: ${missingEvidenceArtifacts.join(", ")}`,
131
141
  });
132
- checks.push(...(await evidenceChecks(runPaths, bundle, cleanupJson)));
142
+ checks.push(...(await evidenceChecks(runPaths, bundle, cleanupRead)));
133
143
  const recordingOk = checks.every((check) => check.ok);
134
144
  if (derivedPublicSafetyFindings.length > 0)
135
145
  checks.push({
@@ -219,7 +229,8 @@ const REQUIRED_SECTIONS = [
219
229
  * Whether run.json, its schema, shape and redaction, and the review artifacts are present. Each
220
230
  * check says what it found, so a failing row never prints the rule it enforces.
221
231
  */
222
- function bundlePresenceChecks(bundle, review) {
232
+ function bundlePresenceChecks(bundleRead, review) {
233
+ const bundle = runJsonValue(bundleRead);
223
234
  const record = isRecord(bundle) ? bundle : undefined;
224
235
  const schema = record?.schema;
225
236
  const schemaOk = schema === RUN_BUNDLE_SCHEMA;
@@ -227,14 +238,26 @@ function bundlePresenceChecks(bundle, review) {
227
238
  const redaction = isRecord(redactionRecord) ? redactionRecord.status : undefined;
228
239
  const missingSections = REQUIRED_SECTIONS.filter((key) => record?.[key] === undefined);
229
240
  const missingReview = [
230
- ...(review.json ? [] : ["review.json is missing or not valid JSON"]),
231
- ...(review.markdown ? [] : ["review.md is missing"]),
241
+ ...(review.json.status === "refused"
242
+ ? [refusalText("review.json", review.json)]
243
+ : runJsonValue(review.json) === null
244
+ ? ["review.json is missing or not valid JSON"]
245
+ : []),
246
+ ...(review.markdown.status === "refused"
247
+ ? [refusalText("review.md", review.markdown)]
248
+ : review.markdown.status === "missing"
249
+ ? ["review.md is missing"]
250
+ : []),
232
251
  ];
233
252
  return [
234
253
  {
235
254
  name: "run.json exists",
236
255
  ok: bundle !== null,
237
- message: bundle === null ? "run.json is missing or not valid JSON" : "run.json is present",
256
+ message: bundleRead.status === "refused"
257
+ ? refusalText(RUN_BUNDLE_FILE, bundleRead)
258
+ : bundle === null
259
+ ? "run.json is missing or not valid JSON"
260
+ : "run.json is present",
238
261
  },
239
262
  {
240
263
  name: "run schema",
@@ -310,7 +333,8 @@ function findingsCheck(name, findings, okMessage, failLabel) {
310
333
  */
311
334
  const SHAPE_UNCHECKED = "not checked, because run.json failed the shape check";
312
335
  /** The bundle-content checks, in report order. A bundle that fails its shape guard has no findings. */
313
- async function evidenceChecks(runPaths, bundle, cleanupJson) {
336
+ async function evidenceChecks(runPaths, bundle, cleanupRead) {
337
+ const cleanupJson = runJsonValue(cleanupRead);
314
338
  const valid = isRunBundle(bundle) ? bundle : null;
315
339
  const pass = (message) => (valid ? message : SHAPE_UNCHECKED);
316
340
  const checks = [
@@ -322,14 +346,17 @@ async function evidenceChecks(runPaths, bundle, cleanupJson) {
322
346
  findingsCheck("shared-world evidence", valid ? sharedWorldEvidenceFindings(valid) : [], pass("live shared-world evidence is well formed for its mode, with single-plane provenance, digest-only checkpoints and attributionLimits, or the run has none"), "shared-world findings"),
323
347
  {
324
348
  name: "cleanup receipt",
325
- ok: cleanupJson === null || (isCleanupResult(cleanupJson) && cleanupJson.ok),
326
- message: cleanupJson === null
327
- ? "no cleanup.json; cleanup was not requested"
328
- : !isCleanupResult(cleanupJson)
329
- ? "cleanup.json is malformed"
330
- : cleanupJson.ok
331
- ? "cleanup.json records a successful cleanup"
332
- : "cleanup.json records a failed cleanup",
349
+ ok: cleanupRead.status !== "refused" &&
350
+ (cleanupJson === null || (isCleanupResult(cleanupJson) && cleanupJson.ok)),
351
+ message: cleanupRead.status === "refused"
352
+ ? `${refusalText("cleanup.json", cleanupRead)}, so verify cannot say how cleanup went`
353
+ : cleanupJson === null
354
+ ? "no cleanup.json; cleanup was not requested"
355
+ : !isCleanupResult(cleanupJson)
356
+ ? "cleanup.json is malformed"
357
+ : cleanupJson.ok
358
+ ? "cleanup.json records a successful cleanup"
359
+ : "cleanup.json records a failed cleanup",
333
360
  },
334
361
  findingsCheck("rerun lineage", valid ? rerunLineageFindings(valid) : [], pass("rerun participants link to their prior status and a fan-out rerun event, or this run is not a rerun"), "rerun lineage findings"),
335
362
  // Cost is advisory on magnitude and fail-closed on labeling/provenance (claims match mechanism).
@@ -371,7 +398,7 @@ function buildShareSafety(args) {
371
398
  });
372
399
  }
373
400
  if (args.unscannedArtifacts.length > 0) {
374
- const paths = [...args.unscannedArtifacts].sort();
401
+ const paths = [...new Set(args.unscannedArtifacts)].sort();
375
402
  const shown = paths.slice(0, MAX_LISTED_UNSCANNED).join(", ");
376
403
  const more = paths.length - MAX_LISTED_UNSCANNED;
377
404
  reasons.push({
@@ -103,10 +103,16 @@ Observation requires a PNG and state signature. It may include bounded URL,
103
103
  title, text, fractional scroll position and heard speech; those remain
104
104
  runtime-only under the existing loop contract. Arbitrary `appState` is refused
105
105
  because v1 has no closed schema for it. The protocol does not truncate strings,
106
- round coordinates, or silently drop unsupported state. A Codex participant's
107
- `humanish_ui` call is checked before it reaches the protocol: a wait longer than 30 seconds is
108
- shortened to 30 seconds there, and the trace records a `wait shortened` notice with both
109
- durations.
106
+ round coordinates, or silently drop unsupported state. A participant may ask for a
107
+ longer wait than one request carries. The computer-use loop shortens it to the study's
108
+ `actor.maxWaitMs` (by default two minutes, or 30 seconds on a desktop with speech, whose heard
109
+ speech reaches the participant only with a screenshot), records a `wait shortened` notice with both
110
+ durations when it does, and sends the rest as consecutive waits of at most 30 seconds. The Codex
111
+ participant's `humanish_ui` schema leaves a wait's length open for that reason; the wire still
112
+ refuses a single wait over 30 seconds. A wait that names no duration, as every OpenAI computer-use
113
+ wait does, reaches the desktop with a length too: the study's `actor.idleWaitMs` (5 seconds by
114
+ default) when the participant's turn only waits or takes screenshots, and 500 ms after an action in
115
+ the same turn.
110
116
 
111
117
  Click, double-click, move, scroll and drag may carry `heldKeys`, which the
112
118
  OpenAI provider maps from its computer tool's `keys`. The Codex participant's
@@ -18,8 +18,13 @@ Progress detection uses the same coarse frame signature as the hosted desktop.
18
18
  The actor receives the existing finite action union. Native tool paths, display,
19
19
  Xauthority and guest temporary directory come from the owner. Key names map to a
20
20
  closed list before reaching xdotool, whose own command syntax must never receive
21
- arbitrary participant strings. A native wheel step is not an exact pixel-scroll
22
- promise; one action permits at most 100 wheel steps. The entire drag is checked
21
+ arbitrary participant strings. The list is `src/guest/desktop-keys.ts`, and the hosted
22
+ E2B desktop translates through it too, so a key name works on both desktops or is
23
+ refused on both. It holds OpenAI's computer-use key names, the names `@e2b/desktop`'s
24
+ `press()` mapped, and every printable ASCII punctuation character as its keysym
25
+ (`?` is `question`). A refused name reaches the participant as a rejected action.
26
+ A native wheel step is not an exact pixel-scroll promise; one action permits at most
27
+ 100 wheel steps. The entire drag is checked
23
28
  before its first input. A pointer action's held keys must be modifiers (Shift,
24
29
  Ctrl, Alt, Super). They go down with one `keydown` before the action and up with
25
30
  one `keyup` after it. When an input fails, the `keyup` is still sent once, unless
@@ -75,10 +75,12 @@ graphically dense (pixel-art) UI. `stableProgressKey(appState)`:
75
75
  ## A single desktop command failure is a recoverable skipped action
76
76
 
77
77
  `execute(action)` runs against a real substrate, and the real `@e2b/desktop`
78
- Sandbox **throws** `CommandExitError` on **any** non-zero exit. A `press`,
78
+ Sandbox **throws** `CommandExitError` on **any** non-zero exit. A keypress,
79
79
  `scroll`, `drag`, `moveMouse`, or `click` can exit non-zero for reasons that have
80
80
  nothing to do with the run's health (a `Ctrl+Minus` zoom keypress exiting `2` was
81
- the reproduced case). The loop treats one such failure as a **recoverable skipped
81
+ the reproduced case). xdotool also skips a key name it cannot resolve with a `No such
82
+ key name` warning and exits `0`; the E2B executor turns that warning into an error with
83
+ exit code `0` and xdotool's stderr. The loop treats one such failure as a **recoverable skipped
82
84
  action**, not a fatal error:
83
85
 
84
86
  - The per-action `execute()` call is wrapped at the loop boundary, in `runActionBatch`
@@ -305,14 +305,16 @@ exactly; this hosted-desktop rule does not change that contract.
305
305
  `subject` is an optional, additive top-level field: structured provenance for
306
306
  what the computer-use, shared-world or scripted route actually drove (code pin plus state story). It
307
307
  is absent on pre-existing bundles and on bundles from routes that have not
308
- adopted it. The field shape, its three sources (`clone`, `app-url`,
309
- `local-tree`), and the `humanish verify` checks that guard it are the schema doc's
308
+ adopted it. The field shape, its four sources (`clone`, `app-url`,
309
+ `local-tree`, `desktop-cli`), and the `humanish verify` checks that guard it are the schema doc's
310
310
  job, not this one: see the `subject` entry under
311
311
  [`schemas.md`](schemas.md#run-bundle). In short, `clone` carries a
312
312
  `repo`/`commit` pin, `local-tree` carries an `archiveSha256`/`dirty` pin
313
- instead (a dirty working tree cannot be commit-pinned), and `app-url` carries
314
- no code pin at all. No path, basename, or other host-machine string ever
315
- enters this field; identity is digests, a sha, a boolean, and counts.
313
+ instead (a dirty working tree cannot be commit-pinned), `desktop-cli` carries the
314
+ declared `product` name, and `app-url` carries no code pin at all. A computer-use run
315
+ of a desktop-cli study saved before `desktop-cli` was recorded says `app-url`, and
316
+ readers accept it. No path, basename, or other host-machine string ever
317
+ enters this field; identity is digests, a sha, a boolean, counts and the declared product name.
316
318
 
317
319
  ## Cost Estimate (advisory)
318
320
 
@@ -723,11 +725,17 @@ a file for secret and path patterns only when the bytes are strict UTF-8 with no
723
725
  control bytes other than tab, line feed and carriage return, the rule bundle
724
726
  export uses for text. `RAW_SCREENSHOTS` and `CONTINUOUS_MEDIA` grade the stream
725
727
  screenshots that actor traces reference under `screenshots/`, where the harness
726
- writes every frame, and the recordings `streams[].recording` registers. Every other file the scan cannot read as text, or cannot read at all,
727
- contributes `UNSCANNED_ARTIFACT`. The reason's message lists the paths, and it keeps
728
- otherwise valid evidence `local_only`. This includes images that only feedback
729
- candidates, adapter artifacts or stream artifact entries cite, and a PNG an actor
730
- trace references outside `screenshots/`. An unregistered
728
+ writes every frame, and the recordings `streams[].recording` registers. Every
729
+ other file the scan cannot read as text, cannot read at all, or that is larger
730
+ than 32 MiB (verify reads at most that much of one file) contributes
731
+ `UNSCANNED_ARTIFACT`. So does a folder the scan cannot list, and the rest of a
732
+ run past its first 10,000 files and folders, the most `export --format bundle`
733
+ takes. The reason's message lists the paths, and it keeps otherwise valid
734
+ evidence `local_only`. This includes images that only feedback candidates,
735
+ adapter artifacts or stream artifact entries cite, and a PNG an actor trace
736
+ references outside `screenshots/`. An adapter artifact larger than 32 MiB grades
737
+ the run `local_only` when nothing cites it, and `blocked` when the bundle cites
738
+ it as evidence, because the evidence check cannot read it. An unregistered
731
739
  `.mp4`, and a file or directory whose name contains `\`, are public-safety
732
740
  findings and block the run.
733
741
 
@@ -76,17 +76,17 @@ map each one to its v3 key.
76
76
  A study declares its route and composes code primitives; it is not a hardcoded kind. The
77
77
  top-level keys:
78
78
 
79
- | Key | Meaning |
80
- | ----------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
81
- | `schema`, `id`, `title`, `description` | the document's identity |
82
- | `route` | required: `preview`, `computer-use`, `shared-world`, `terminal` or `scripted` |
83
- | `mode` | `dry-run` (the default) or `live` |
84
- | `actor` | who drives it, one object: `type`, `model`, `localAgent`, `persona`, `mission`, `tasks`, `maxOutputTokens`, `reasoningEffort`, `stopWhen`, `dwell` |
85
- | `participants` | a count, a homogeneous `{ count, instruction }`, or a list of entries (below) |
86
- | `surfaces` | scripted only: `[desktop]` or `[desktop, mobile]` |
87
- | `caps` | the route's budget (below) |
88
- | `scenario` | scripted only: a committed scenario id or path |
89
- | `subject`, `execution`, `policies`, `review`, `defaults`, `comms` | as below |
79
+ | Key | Meaning |
80
+ | ----------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
81
+ | `schema`, `id`, `title`, `description` | the document's identity |
82
+ | `route` | required: `preview`, `computer-use`, `shared-world`, `terminal` or `scripted` |
83
+ | `mode` | `dry-run` (the default) or `live` |
84
+ | `actor` | who drives it, one object: `type`, `model`, `localAgent`, `persona`, `mission`, `tasks`, `maxOutputTokens`, `maxWaitMs`, `idleWaitMs`, `reasoningEffort`, `stopWhen`, `dwell` |
85
+ | `participants` | a count, a homogeneous `{ count, instruction }`, or a list of entries (below) |
86
+ | `surfaces` | scripted only: `[desktop]` or `[desktop, mobile]` |
87
+ | `caps` | the route's budget (below) |
88
+ | `scenario` | scripted only: a committed scenario id or path |
89
+ | `subject`, `execution`, `policies`, `review`, `defaults`, `comms` | as below |
90
90
 
91
91
  The parser reads only the declared route's keys. A key that route never reads is an error that
92
92
  names the route, such as "route: computer-use does not read scenario. Remove it.", and an
@@ -317,9 +317,9 @@ The rest of this section names keys by their v3 spelling:
317
317
  fail-closed cap in force. `maxMinutes` is the
318
318
  wall-clock kill; `maxUsd`/`maxJobs` are enforced fail-closed against the cost
319
319
  ledger after the session (a run whose known spend exceeds the cap fails closed,
320
- `HUMANISH_TERMINAL_CAPS_EXCEEDED`). Core records Codex provider spend as
321
- unpriced tokens and measures no product, media or payment spend, so no line can trip a
322
- positive `maxUsd`: a live run refuses `maxUsd > 0` before creating a sandbox
320
+ `HUMANISH_TERMINAL_CAPS_EXCEEDED`). Core prices the Codex participant's tokens as an
321
+ estimate that the cap does not count, and measures no product, media or payment spend, so no
322
+ line can trip a positive `maxUsd`: a live run refuses `maxUsd > 0` before creating a sandbox
323
323
  (`HUMANISH_TERMINAL_UNPRICED_CAP`). A terminal `maxUsd` cap has no adopter cost source
324
324
  until [issue 347](https://github.com/danielgwilson/humanish/issues/347) lands;
325
325
  `StudyDeps.costProbe` is a test seam. The sandbox's server-side timeout is the steps before the
@@ -529,8 +529,8 @@ Core-owned fields:
529
529
  an older run, else `inferLegacyStudyId`, which reads the `persona.source` or `scenario.source`
530
530
  convention `study:<id>`, or `lab:<id>` from runs written before 0.108, and nothing else.
531
531
  - `subject` (optional, additive): structured subject provenance,
532
- `{ source: clone | app-url | local-tree, repo?, commit?, archiveSha256?,
533
- dirty?, envNames?, state }` where `state` is `{ provenance: seeded |
532
+ `{ source: clone | app-url | local-tree | desktop-cli, repo?, product?, commit?,
533
+ archiveSha256?, dirty?, envNames?, state }` where `state` is `{ provenance: seeded |
534
534
  unpinned | declared-not-run | undeclared | external-public, seed?: [{ name, when,
535
535
  commandDigest, ok?, exitCode?, timedOut?, durationMs? }], externalEnvNames?
536
536
  }`. Emitted by the computer-use and shared-world routes and the clone
@@ -538,7 +538,10 @@ Core-owned fields:
538
538
  (64-hex sha256, the local-tree provenance pin) and `dirty` (host git
539
539
  porcelain status at pack time) are local-tree-route fields, additive under
540
540
  `humanish.run-bundle.v1`: a dirty working tree cannot be commit-pinned, so
541
- the archive content digest stands in for it. `commandDigest` is the
541
+ the archive content digest stands in for it. `product` is the desktop-cli field: the
542
+ `subject.product.name` the study declares, with state `undeclared`. A computer-use
543
+ run of a desktop-cli study saved before this field existed records `app-url` with no
544
+ `product`. `commandDigest` is the
542
545
  sha256-16 of the exact seed command; command text and env values never
543
546
  appear. `humanish verify` fails closed when a live `local-tree` bundle carries
544
547
  no well-formed `archiveSha256`, in addition to the existing `subject state
@@ -982,6 +985,14 @@ Core-owned fields:
982
985
  “provider reply cut off by the output limit; asking again” notice. A second
983
986
  cut-off reply to the same request ends the session with
984
987
  `stopCause: provider_output_limit`.
988
+ - optional `waitSettings`: the wait lengths a computer-use participant's session
989
+ applied, in milliseconds, defaults included. `maxWaitMs` is the longest one
990
+ wait action lasts (`actor.maxWaitMs`); a longer wait is shortened to it.
991
+ `idleWaitMs` is how long a wait that names no duration lasts in a turn that
992
+ only waits or takes screenshots (`actor.idleWaitMs`), and `settleWaitMs` how
993
+ long such a wait lasts after an action in the same turn. Each `ui_action` for
994
+ a wait is titled with the length it lasted. Absent on other routes and on
995
+ earlier bundles
985
996
  - optional `conversation`: how the first-party OpenAI computer-use provider
986
997
  carried the conversation between requests. `mode` is `threaded` (the server
987
998
  kept it through `previous_response_id`) or `explicit_context` (the server kept
@@ -1142,15 +1153,20 @@ discipline** that distinguishes three states and never conflates them:
1142
1153
 
1143
1154
  `knownTotalUsd` sums only the non-null lines (a `null` line contributes nothing
1144
1155
  and is never coerced to `0`); `fullyMeasured` is true only when no line is null.
1145
- Core meters only the `provider` line, populated from the actor trace's
1146
- `tokenUsage.costUsd` when present (else `null`); `product`/`media`/`payment`
1156
+ Core meters only the `provider` line, from the actor trace's `tokenUsage.costUsd` when present
1157
+ (the Codex usage records the terminal route reads carry none today); `product`/`media`/`payment`
1147
1158
  remain `null`, because no adopter can supply those signals until
1148
- [issue 347](https://github.com/danielgwilson/humanish/issues/347) lands. The terminal route records the model as `codex` and does not pin Codex's
1149
- model, so a measured token count stays `usd: null` with `source: unpriced-token-usage`, and
1150
- `knownTotalUsd: 0` with `fullyMeasured: false` means no line of this ledger was priced. E2B
1151
- time for the terminal sandbox is not a line of this ledger, whose lines are checked against
1152
- `caps.maxUsd`. The run bundle's `cost` summary prices it as a `desktop-minutes`
1153
- line (see Run Cost Summary And Estimated Actor Cost below).
1159
+ [issue 347](https://github.com/danielgwilson/humanish/issues/347) lands. Without a measured
1160
+ charge the `provider` line keeps `usd: null` and counts the participant's tokens. The route
1161
+ passes Codex a model and prices the tokens once from it (the actor trace's `estimatedCost`); the
1162
+ line carries that price as `estimatedUsd` with `source: estimated-token-usage`, the same figure as
1163
+ the run bundle's `model-tokens` cost line. The estimate is not part of `knownTotalUsd`, so
1164
+ `caps.maxUsd` and the no-spend proof do not count it. Tokens of a model with no rate stay
1165
+ `source: unpriced-token-usage` with the model named in the note, and a run whose output had no
1166
+ usage record has `source: unmeasured`. `knownTotalUsd: 0` with `fullyMeasured: false` means no
1167
+ line of this ledger was measured. E2B time for the terminal sandbox is not a line of this ledger,
1168
+ whose lines are checked against `caps.maxUsd`. The run bundle's `cost` summary prices it as a
1169
+ `desktop-minutes` line (see Run Cost Summary And Estimated Actor Cost below).
1154
1170
 
1155
1171
  ```yaml
1156
1172
  schema: humanish.terminal-cost-ledger.v1
@@ -1159,7 +1175,8 @@ lines:
1159
1175
  product: { usd: null, count: null, source: unmeasured, note: "…no signal yet…" }
1160
1176
  media: { usd: null, count: null, source: unmeasured, note: "…no signal yet…" }
1161
1177
  payment: { usd: null, count: null, source: unmeasured, note: "…no signal yet…" }
1162
- provider: { usd: null, source: unmeasured, note: "…no tokenUsage.costUsd this run…" }
1178
+ provider:
1179
+ { usd: null, estimatedUsd: 0.37157, source: estimated-token-usage, note: "…estimated at…" }
1163
1180
  knownTotalUsd: 0
1164
1181
  fullyMeasured: false
1165
1182
  ```
@@ -1171,9 +1188,9 @@ it explicitly cannot vouch for). `satisfied` is true only when every known line
1171
1188
  is within `maxUsd` (for a no-spend run, `maxUsd: 0` ⇒ every known line is `0`).
1172
1189
  Unmeasured lines never break it, so a ledger whose lines are all `null` is
1173
1190
  `satisfied` with nothing proven. The `statement` says which case applies: it
1174
- names the measured lines and their dollars, provider tokens that were counted
1175
- but unpriced (with the counts), and the lines with no signal; when no line was
1176
- measured it says the proof was not established. A proof never claims zero on a
1191
+ names the measured lines and their dollars, the provider tokens that were counted
1192
+ (with the counts, and their estimate or that they are unpriced), and the lines with
1193
+ no signal; when no line was measured it says the proof was not established. A proof never claims zero on a
1177
1194
  `null` line; verification fails closed if it does.
1178
1195
 
1179
1196
  ```yaml
@@ -205,7 +205,16 @@ analysis, and treats an analysis that breaks them as unavailable.
205
205
 
206
206
  The narrative scrub removes a run's known transient values, such as a received one-time code or
207
207
  link, from every generated text field. It finds each value as written and in its encoded forms:
208
- percent-encoded, JSON-escaped, base64, base64url and hex, and where escapes split it. A field
208
+ percent-encoded, JSON-escaped, base64, base64url and hex, and where escapes split it. Base64 and
209
+ hex forms are searched from 6 characters. A value of 4 bytes, such as the code `7439`, is found
210
+ in base64 written whole (`NzQzOQ` or `NzQzOQ==`) and not inside a longer base64 run, where 4 or 5
211
+ characters of its encoding do not depend on the bytes around it;
212
+ `tests/evidence/secret-scrub.test.ts` records the case. The cost of the short forms is
213
+ over-redaction: an identifier that holds a short value's base64 loses that part, as
214
+ `aMTIzNAz` becomes `a[REDACTED_SECRET]z` while the code `1234` (`MTIzNA`) is registered, since a
215
+ match cannot tell whether the identifier came from the value. In the text of 9 real runs, no 6-
216
+ or 7-character form of any 4-, 5- or 6-digit code occurred. Run evidence is scrubbed for the same
217
+ forms. A field
209
218
  with a value is stored decoded with the value replaced; a field without one keeps its spelling. A
210
219
  value inside a bracketed span, such as `[REDACTED_373433393231]`, is found too: only the markers
211
220
  humanish writes (`[REDACTED_SECRET]`, `[REDACTED_LOCAL_PATH]`, `[REDACTED_RUNTIME_PATH]`,
@@ -102,7 +102,9 @@ accepted. Known usage is retained; missing usage remains unknown.
102
102
 
103
103
  The CLI's JSON keeps `runOk` for the route's own result, `automaticAnalysis`
104
104
  for post-run analysis, and `ok` for the overall request. Failed, cancelled or
105
- unknown analysis produces exit code 2 without discarding the recording. A missing
105
+ unknown analysis produces exit code 2 without discarding the recording. When the
106
+ run's own result is ok, the first line of the human output reads
107
+ `humanish run <study>: live run finished; the analysis did not complete`. A missing
106
108
  `OPENAI_API_KEY` skips default analysis and preserves a successful run exit;
107
109
  `automaticAnalysisTrigger: "default"` distinguishes that case in JSON. So does a
108
110
  default analysis whose expected cost, plus the margin, is over the default $3 cap:
@@ -144,5 +146,9 @@ outcomes and the deterministic review verdict are never rewritten by analysis.
144
146
  | `AUTOMATIC_ANALYSIS_CANCELLATION_UNAVAILABLE` | `unknown` | A cancellation could not be recorded |
145
147
  | `AUTOMATIC_ANALYSIS_OUTCOME_UNKNOWN` | `unknown` | The outcome could not be determined |
146
148
 
149
+ A run refused before it was created (`run: not-created`) prints no `analysis:` line, since there
150
+ is no run to analyze and the refusal says why. Its JSON keeps the record, with
151
+ `AUTOMATIC_ANALYSIS_SOURCE_UNAVAILABLE`, or `AUTOMATIC_ANALYSIS_DRY_RUN` for a dry run.
152
+
147
153
  See the [analysis contract](../contracts/study-analysis.md) for selection limits,
148
154
  evidence validation, actual usage, corrections and share-safe export behavior.