humanish 0.124.0 → 0.126.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +31 -0
- package/CONTEXT.md +1 -1
- package/README.md +2 -2
- package/dist/actors/computer-use/actor.d.ts +2 -0
- package/dist/actors/computer-use/actor.js +1 -0
- package/dist/actors/computer-use/loop/clock.d.ts +9 -8
- package/dist/actors/computer-use/loop/clock.js +27 -17
- package/dist/actors/computer-use/loop/debrief.d.ts +1 -1
- package/dist/actors/computer-use/loop/debrief.js +15 -5
- package/dist/actors/computer-use/loop/ending.d.ts +8 -5
- package/dist/actors/computer-use/loop/ending.js +8 -5
- package/dist/actors/computer-use/loop/observation.d.ts +6 -0
- package/dist/actors/computer-use/loop/observation.js +19 -1
- package/dist/actors/computer-use/loop/session.d.ts +4 -2
- package/dist/actors/computer-use/loop/session.js +4 -1
- package/dist/actors/computer-use/loop/trace.js +4 -2
- package/dist/actors/computer-use/loop/types.d.ts +22 -8
- package/dist/actors/computer-use/loop.js +5 -5
- package/dist/actors/contract.d.ts +31 -8
- package/dist/actors/contract.js +3 -0
- package/dist/actors/stop-cause.js +1 -0
- package/dist/analysis/evidence.js +2 -2
- package/dist/cli/commands/study.js +8 -0
- package/dist/evidence/redaction.d.ts +1 -0
- package/dist/evidence/redaction.js +1 -0
- package/dist/evidence/trace-page.d.ts +6 -0
- package/dist/evidence/trace-page.js +90 -0
- package/dist/index.d.ts +1 -0
- package/dist/index.js +1 -0
- package/dist/observer/feed-cursors.d.ts +14 -0
- package/dist/observer/feed-cursors.js +61 -0
- package/dist/observer/participant-items.d.ts +7 -0
- package/dist/observer/pinned-files.d.ts +4 -1
- package/dist/observer/pinned-files.js +10 -7
- package/dist/observer/run-routes.d.ts +1 -1
- package/dist/observer/run-routes.js +39 -17
- package/dist/observer-app.html +7 -7
- package/dist/routes/computer-use/bundle.js +4 -0
- package/dist/routes/computer-use/diagnostics.d.ts +1 -1
- package/dist/routes/computer-use/diagnostics.js +3 -3
- package/dist/routes/computer-use/e2b-desktop/desktop.js +2 -1
- package/dist/routes/computer-use/e2b-desktop/prepare.d.ts +6 -0
- package/dist/routes/computer-use/e2b-desktop/prepare.js +47 -1
- package/dist/routes/computer-use/e2b-desktop/start.d.ts +2 -1
- package/dist/routes/computer-use/e2b-desktop/start.js +15 -3
- package/dist/routes/computer-use/e2b-desktop/state.d.ts +3 -1
- package/dist/routes/computer-use/e2b-desktop/state.js +2 -0
- package/dist/routes/computer-use/fanout-records.js +4 -0
- package/dist/routes/computer-use/live-flush.d.ts +9 -4
- package/dist/routes/computer-use/live-flush.js +16 -3
- package/dist/routes/computer-use/participant-deps.d.ts +1 -1
- package/dist/routes/computer-use/participant-desktop.d.ts +1 -1
- package/dist/routes/computer-use/participant-model.js +16 -6
- package/dist/routes/computer-use/participant-runs.d.ts +2 -2
- package/dist/routes/computer-use/participant-runs.js +11 -7
- package/dist/routes/computer-use/plan.js +39 -1
- package/dist/routes/computer-use/setup.d.ts +1 -0
- package/dist/routes/computer-use/setup.js +28 -0
- package/dist/routes/computer-use/single-bundle.d.ts +5 -1
- package/dist/routes/computer-use/single-bundle.js +2 -0
- package/dist/routes/computer-use/types.d.ts +16 -10
- package/dist/routes/shared-world/bundle-records.js +3 -0
- package/dist/routes/shared-world/participant-specs.js +3 -2
- package/dist/routes/shared-world/plan.js +5 -0
- package/dist/run/bundle-shape.js +2 -2
- package/dist/run/bundle.d.ts +10 -3
- package/dist/run/bundle.js +11 -2
- package/dist/run/contained-output.d.ts +26 -0
- package/dist/run/contained-output.js +101 -1
- package/dist/run/evidence-files.d.ts +12 -1
- package/dist/run/evidence-files.js +70 -40
- package/dist/run/judge.d.ts +7 -4
- package/dist/run/judge.js +13 -8
- package/dist/run/live-items.d.ts +67 -0
- package/dist/run/live-items.js +262 -0
- package/dist/run/locate.d.ts +10 -0
- package/dist/run/locate.js +10 -0
- package/dist/run/run.d.ts +3 -2
- package/dist/run/run.js +29 -8
- package/dist/run/sandbox-ids.d.ts +0 -5
- package/dist/run/sandbox-ids.js +0 -7
- package/dist/run/stream-trace.d.ts +40 -10
- package/dist/run/stream-trace.js +151 -22
- package/dist/run/streams.d.ts +33 -2
- package/dist/study/arrivals.d.ts +32 -14
- package/dist/study/arrivals.js +47 -21
- package/dist/study/bundle-estimate.d.ts +6 -5
- package/dist/study/bundle-estimate.js +12 -7
- package/dist/study/desktop-files.d.ts +37 -0
- package/dist/study/desktop-files.js +174 -0
- package/dist/study/keys.d.ts +8 -0
- package/dist/study/keys.js +3 -0
- package/dist/study/migrate/v2.js +12 -3
- package/dist/study/parse/actors.d.ts +1 -1
- package/dist/study/parse/actors.js +51 -26
- package/dist/study/parse/clock.d.ts +18 -0
- package/dist/study/parse/clock.js +36 -0
- package/dist/study/parse/execution.js +6 -0
- package/dist/study/plan-base.js +4 -1
- package/dist/study/plan-participants.d.ts +19 -5
- package/dist/study/plan-participants.js +17 -1
- package/dist/study/plan.js +24 -13
- package/dist/study/preflight.js +4 -0
- package/dist/study/types.d.ts +34 -5
- package/dist/study/validation.js +3 -0
- package/dist/study/warnings.d.ts +2 -2
- package/dist/study/warnings.js +7 -0
- package/dist/subject/local-tree-archive.d.ts +7 -0
- package/dist/subject/local-tree-archive.js +12 -0
- package/dist/substrates/e2b/desktop-browser.d.ts +4 -1
- package/dist/substrates/e2b/desktop-browser.js +6 -2
- package/dist/substrates/e2b/desktop-clock.d.ts +10 -0
- package/dist/substrates/e2b/desktop-clock.js +29 -0
- package/dist/substrates/e2b/sdk.d.ts +6 -0
- package/dist/verify/artifacts.js +39 -51
- package/dist/verify/pages.d.ts +7 -0
- package/dist/verify/pages.js +50 -0
- package/dist/verify/screenshot-reads.d.ts +21 -0
- package/dist/verify/screenshot-reads.js +46 -0
- package/dist/verify/verify.js +17 -9
- package/docs/architecture/actor-contract.md +2 -0
- package/docs/architecture/external-public-shared-world.md +4 -1
- package/docs/architecture/github-feedback-loop.md +1 -1
- package/docs/architecture/observer.md +10 -1
- package/docs/contracts/README.md +1 -1
- package/docs/contracts/policy.md +7 -0
- package/docs/contracts/run-bundle.md +106 -14
- package/docs/contracts/schemas.md +110 -40
- package/package.json +4 -2
package/dist/verify/verify.js
CHANGED
|
@@ -5,7 +5,7 @@ import { analysisSharingProblems } from "../analysis/sharing.js";
|
|
|
5
5
|
import { containsSensitive, REDACTED_SANDBOX_ID } from "../evidence/redaction.js";
|
|
6
6
|
import { keyedSandboxIds, readRunSandboxIds } from "../run/sandbox-ids.js";
|
|
7
7
|
import { validatePreparedRunArtifactPaths } from "../run/paths.js";
|
|
8
|
-
import { RUN_BUNDLE_FILE, RUN_BUNDLE_SCHEMA, RUN_BUNDLE_V1_SCHEMA, } from "../run/bundle.js";
|
|
8
|
+
import { isReadRunBundleSchema, RUN_BUNDLE_FILE, RUN_BUNDLE_SCHEMA, RUN_BUNDLE_V1_SCHEMA, RUN_BUNDLE_V2_SCHEMA, } from "../run/bundle.js";
|
|
9
9
|
import { ContainedReadRefusedError, refusalText } from "../run/contained-output.js";
|
|
10
10
|
import { isCleanupResult, isRunBundle } from "../run/bundle-shape.js";
|
|
11
11
|
import { loadStreamTraces } from "../run/stream-trace.js";
|
|
@@ -16,6 +16,7 @@ import { runNotFoundMessage } from "../run/run-not-found.js";
|
|
|
16
16
|
import { actorVerdictConsistencyFindings, noEngagementActorFindings, validateCodexAppServerEvidence, validateTerminalProductEvidence, } from "./actor.js";
|
|
17
17
|
import { invalidRunEvidenceReferences, MAX_REPORTED_FINDINGS, missingLocalEvidenceArtifacts, rawScreenshotPostureWarnings, rawScreenshotStreamIds, redactedShapeFramePaths, scanRunPublicSafetyArtifacts, scanSourceEvidence, streamScreenshotPaths, unscannedMessage, } from "./artifacts.js";
|
|
18
18
|
import { flatContextWarnings } from "./context-growth.js";
|
|
19
|
+
import { pageFindings } from "./pages.js";
|
|
19
20
|
import { runNotFinished } from "./liveness.js";
|
|
20
21
|
import { costAndReceiptFindings } from "./costs.js";
|
|
21
22
|
import { rerunLineageFindings } from "./rerun.js";
|
|
@@ -229,7 +230,7 @@ function evidenceArtifactsCheck(shaped, problems) {
|
|
|
229
230
|
async function loadTraces(runPaths, stored) {
|
|
230
231
|
const scanned = { sensitive: [], opaque: [] };
|
|
231
232
|
if (!isRunBundle(stored))
|
|
232
|
-
return { bundle: stored, refusedTraces: [], ...scanned };
|
|
233
|
+
return { bundle: stored, refusedTraces: [], ...scanned, pages: [] };
|
|
233
234
|
// Each trace is source evidence: scanned by the bytes it was joined from, apart from the
|
|
234
235
|
// directory walk and any path exemption the walk makes.
|
|
235
236
|
const { bundle, refused } = await loadStreamTraces(runPaths, stored, {
|
|
@@ -239,9 +240,14 @@ async function loadTraces(runPaths, stored) {
|
|
|
239
240
|
scanned[scan === "sensitive" ? "sensitive" : "opaque"].push(file);
|
|
240
241
|
},
|
|
241
242
|
});
|
|
242
|
-
return {
|
|
243
|
+
return {
|
|
244
|
+
bundle,
|
|
245
|
+
refusedTraces: refused.map((error) => error.message),
|
|
246
|
+
...scanned,
|
|
247
|
+
pages: pageFindings(bundle),
|
|
248
|
+
};
|
|
243
249
|
}
|
|
244
|
-
/** The trace scan's results, beside the walk's: each file named once. */
|
|
250
|
+
/** The trace scan's results, beside the walk's: each file named once, then each page finding. */
|
|
245
251
|
function addTraceScan(traces, findings, unscanned) {
|
|
246
252
|
for (const file of traces.sensitive)
|
|
247
253
|
if (!findings.includes(`sensitive text ${file}`))
|
|
@@ -249,6 +255,7 @@ function addTraceScan(traces, findings, unscanned) {
|
|
|
249
255
|
for (const file of traces.opaque)
|
|
250
256
|
if (!unscanned.files.includes(file))
|
|
251
257
|
unscanned.files.push(file);
|
|
258
|
+
findings.push(...traces.pages);
|
|
252
259
|
}
|
|
253
260
|
/** The top-level run.json sections the shape guard requires. */
|
|
254
261
|
const REQUIRED_SECTIONS = [
|
|
@@ -272,7 +279,7 @@ function bundlePresenceChecks(bundleRead, review) {
|
|
|
272
279
|
const bundle = runJsonValue(bundleRead);
|
|
273
280
|
const record = isRecord(bundle) ? bundle : undefined;
|
|
274
281
|
const schema = record?.schema;
|
|
275
|
-
const schemaOk = schema
|
|
282
|
+
const schemaOk = isReadRunBundleSchema(schema);
|
|
276
283
|
const redactionRecord = record?.redaction;
|
|
277
284
|
const redaction = isRecord(redactionRecord) ? redactionRecord.status : undefined;
|
|
278
285
|
const missingSections = REQUIRED_SECTIONS.filter((key) => record?.[key] === undefined);
|
|
@@ -305,7 +312,7 @@ function bundlePresenceChecks(bundleRead, review) {
|
|
|
305
312
|
? `run.json declares ${String(schema)}`
|
|
306
313
|
: record === undefined
|
|
307
314
|
? "no readable run.json, so no schema"
|
|
308
|
-
: `run.json declares ${typeof schema === "string" ? schema : "no schema"}; verify reads ${RUN_BUNDLE_SCHEMA} and ${RUN_BUNDLE_V1_SCHEMA}`,
|
|
315
|
+
: `run.json declares ${typeof schema === "string" ? schema : "no schema"}; verify reads ${RUN_BUNDLE_SCHEMA}, ${RUN_BUNDLE_V2_SCHEMA} and ${RUN_BUNDLE_V1_SCHEMA}`,
|
|
309
316
|
},
|
|
310
317
|
{
|
|
311
318
|
name: "run bundle shape",
|
|
@@ -439,10 +446,11 @@ function buildShareSafety(args) {
|
|
|
439
446
|
const unscanned = unscannedMessage(args.unscannedArtifacts);
|
|
440
447
|
if (unscanned !== undefined)
|
|
441
448
|
reasons.push({ code: "UNSCANNED_ARTIFACT", message: unscanned });
|
|
442
|
-
//
|
|
443
|
-
// a participant was still working (liveActor)
|
|
449
|
+
// From v2, run.json binds each finished participant's items by its trace index. Items written
|
|
450
|
+
// while a participant was still working (liveActor) belong to no finished trace, so such a run
|
|
451
|
+
// stays local.
|
|
444
452
|
const live = args.bundle.streams.filter((stream) => stream.liveActor !== undefined);
|
|
445
|
-
if (args.bundle.schema
|
|
453
|
+
if (args.bundle.schema !== RUN_BUNDLE_V1_SCHEMA && live.length > 0)
|
|
446
454
|
reasons.push({
|
|
447
455
|
code: "RUN_NOT_FINISHED",
|
|
448
456
|
message: `The run did not finish: run.json holds in-progress items for ${live.map((stream) => stream.id).join(", ")}, which no trace file binds. Review it locally; rerun the study for evidence you can share.`,
|
|
@@ -167,6 +167,7 @@ export interface ActorTraceItem {
|
|
|
167
167
|
command?: { text?: string; cwd?: string; exitCode?: number; outputTail?: string };
|
|
168
168
|
screenshotRef?: { path: string; redaction: "blurred" | "ocr_scrubbed" | "none" };
|
|
169
169
|
text?: string; // redacted
|
|
170
|
+
page?: { origin: string; path: string }; // computer-use screenshot items; redacted, no query
|
|
170
171
|
}
|
|
171
172
|
|
|
172
173
|
export interface ActorCapabilities {
|
|
@@ -204,6 +205,7 @@ export interface ActorTrace {
|
|
|
204
205
|
ids: { sessionId?: string; threadId?: string; turnId?: string; model?: string };
|
|
205
206
|
counts: Record<string, number>;
|
|
206
207
|
items: ActorTraceItem[];
|
|
208
|
+
finalPage?: { origin: string; path: string }; // computer-use: the last observation's page
|
|
207
209
|
tokenUsage?: { input?: number; output?: number; total?: number; costUsd?: number };
|
|
208
210
|
estimatedCost?: ActorEstimatedCost; // humanish.actor-estimated-cost.v1 (additive)
|
|
209
211
|
capabilities: ActorCapabilities;
|
|
@@ -120,7 +120,10 @@ The runtime `location.href` (and the 6-char code inside it) is never persisted r
|
|
|
120
120
|
runtime-only; the threaded code flows only into the follower's composed prompt (never a raw bundle
|
|
121
121
|
field) and is scrubbed from all narration once latched; the shared origin and lobby path persist only
|
|
122
122
|
as sha256-16 digests (`publicOriginDigest`, `lobbyConvergenceDigest`). This matches the existing
|
|
123
|
-
e2b-URL / host-digest redaction discipline.
|
|
123
|
+
e2b-URL / host-digest redaction discipline. The computer-use loop records each observation's page
|
|
124
|
+
origin and path by default; the shared-world runner (`participantRunDeps` in
|
|
125
|
+
`src/routes/shared-world/participant-specs.ts`) runs every session with `recordPages: false`, on
|
|
126
|
+
both planes, so no page and no `finalPage` reaches a shared-world trace.
|
|
124
127
|
|
|
125
128
|
## Mobile fidelity caveat
|
|
126
129
|
|
|
@@ -129,7 +129,7 @@ humanish_swarm:
|
|
|
129
129
|
- .env*
|
|
130
130
|
- .github/workflows/**
|
|
131
131
|
- infra/**
|
|
132
|
-
artifact_schema_version: humanish.run-bundle.
|
|
132
|
+
artifact_schema_version: humanish.run-bundle.v3
|
|
133
133
|
credential_manifest: []
|
|
134
134
|
network_policy: no_network
|
|
135
135
|
spend_policy: no_spend
|
|
@@ -74,7 +74,16 @@ public-safe metadata.
|
|
|
74
74
|
|
|
75
75
|
The browser polls `observer-feed.json` with `no-store` caching: the server's
|
|
76
76
|
projection of the run as `{ data, items }`, observer-data.json's content and the
|
|
77
|
-
items, from one read of the run per poll.
|
|
77
|
+
items, from one read of the run per poll. While a participant runs, its entry in
|
|
78
|
+
`items` carries `live`, a cursor `<position>.<generation>.<items>.<bytes>` naming its
|
|
79
|
+
live file (`live/<position>.<generation>.ndjson`) and how far the list goes. The page
|
|
80
|
+
sends the cursors of its last body back as `?after=<cursor>,<cursor>`, and the server
|
|
81
|
+
then reads and sends only the items past each one, marking that entry with
|
|
82
|
+
`liveAfter`, the cursor they follow (`src/observer/feed-cursors.ts`). A cursor for an
|
|
83
|
+
earlier generation, past the index or off a line end gets the participant's full list,
|
|
84
|
+
and so does every participant of a finished run. A tail is checked for line ends and
|
|
85
|
+
its line count; the sha256 run.json records is checked whenever a list is read whole.
|
|
86
|
+
A page that cannot join a body sends no cursors with its next poll. Static
|
|
78
87
|
`file://` opening still works for immutable review, but follow mode is the
|
|
79
88
|
operator path. Agents and CI should use `humanish watch --json --no-open` for
|
|
80
89
|
the same fresh evidence without browser open or a long-running process.
|
package/docs/contracts/README.md
CHANGED
|
@@ -8,7 +8,7 @@ disagree.
|
|
|
8
8
|
(`RunStudyOptions`) and the exports removed in 0.107.0.
|
|
9
9
|
- [core.md](core.md): the records a run writes under `.humanish/runs/<id>/` and how each route
|
|
10
10
|
starts a run.
|
|
11
|
-
- [run-bundle.md](run-bundle.md): `humanish.run-bundle.
|
|
11
|
+
- [run-bundle.md](run-bundle.md): `humanish.run-bundle.v3`, the evidence contract that `verify`
|
|
12
12
|
checks.
|
|
13
13
|
- [policy.md](policy.md): credential, network, spend, redaction, sharing and assisted-run
|
|
14
14
|
boundaries.
|
package/docs/contracts/policy.md
CHANGED
|
@@ -146,6 +146,13 @@ certification that the artifact is free of every class in the policy below. A
|
|
|
146
146
|
first-class PII/PHI detector is [planned](https://github.com/danielgwilson/humanish/issues/108) and would move the listed
|
|
147
147
|
PII classes from author-responsibility to enforced.
|
|
148
148
|
|
|
149
|
+
The same scan reads the page each computer-use observation recorded (`items[].page`,
|
|
150
|
+
`finalPage`). verify also reports a page whose path holds a query string or fragment, or whose
|
|
151
|
+
origin holds user info or anything after the host, since such a part can carry a short token no
|
|
152
|
+
pattern matches. The recorder withholds email- and token-shaped path segments
|
|
153
|
+
([schemas.md](schemas.md#actor-trace)); a name or other personal data in a readable path is not
|
|
154
|
+
detected.
|
|
155
|
+
|
|
149
156
|
Required redaction gates:
|
|
150
157
|
|
|
151
158
|
- run bundle verification;
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Run Bundle Contract
|
|
2
2
|
|
|
3
|
-
`humanish.run-bundle.
|
|
3
|
+
`humanish.run-bundle.v3` is the evidence contract. The TypeScript shape in `src/run/bundle.ts`
|
|
4
4
|
(with `streams[]` in `src/run/streams.ts` and `sharedWorld` as `SharedWorldEvidence` in
|
|
5
5
|
`src/run/shared-world-evidence.ts`) and the fail-closed checks in `src/verify/verify.ts` are the
|
|
6
6
|
source of truth. This page explains the stable public fields and the extension rules; it does not
|
|
@@ -46,7 +46,11 @@ impressions, token usage and estimated cost. Each trace is written once.
|
|
|
46
46
|
`src/run/stream-shape.ts` holds the rule, and the writer, the shape check and the loader all use
|
|
47
47
|
it. Any other path, such as `observer/index.html`, fails the shape check.
|
|
48
48
|
|
|
49
|
-
|
|
49
|
+
A reader outside humanish loads a run through the package's `loadRunBundle(cwd, run)`, with `run`
|
|
50
|
+
a run id or `latest`: it returns run.json with every trace file and live file joined and checked
|
|
51
|
+
as below, for v1, v2 and v3 bundles, and throws an Error named `StreamTraceRefusedError` for a
|
|
52
|
+
file that does not match its index. Inside humanish, readers load a stream through
|
|
53
|
+
`loadStreamTrace(runPaths, stream)`. A v1 stream returns as it is.
|
|
50
54
|
An indexed stream's file is read under the 32 MiB per-file limit (analysis uses its 16 MiB
|
|
51
55
|
source limit for each trace) and refused unless its size, sha256, item count and screenshot count
|
|
52
56
|
match the index. A trace file belongs to one participant: every stream whose index names a file
|
|
@@ -60,12 +64,69 @@ apart from its walk of the run folder, so no path exemption the walk makes appli
|
|
|
60
64
|
run.json's digest, which an analysis record binds, covers each index and so each trace.
|
|
61
65
|
|
|
62
66
|
Mid-run flushes: the writer indexes every finished participant on every write, flushes included,
|
|
63
|
-
so a
|
|
64
|
-
`streams[].liveActor
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
67
|
+
so a run.json from v2 on never holds `actor.items`. A participant still working has its items so
|
|
68
|
+
far in `streams[].liveActor`. A run.json may carry `liveActor` only while its run has not
|
|
69
|
+
finished: one whose `outcome.state` is `finished` and still holds `liveActor` is refused. An
|
|
70
|
+
unfinished run.json with `liveActor` (a flush, or the last write of an interrupted run) verifies
|
|
71
|
+
as `local_only` with the share-safety reason `RUN_NOT_FINISHED`, never `share_ready`. v1 bundles
|
|
72
|
+
keep their earlier behavior.
|
|
73
|
+
|
|
74
|
+
## Live items
|
|
75
|
+
|
|
76
|
+
From v3, a running participant's items so far are in a live file, so a flush appends only the
|
|
77
|
+
items recorded since the last one instead of rewriting run.json with every item. The run writer
|
|
78
|
+
(`liveItemsWriter` in `src/run/live-items.ts`, called from `src/run/run.ts`) keeps, for each
|
|
79
|
+
stream with a `liveActor`:
|
|
80
|
+
|
|
81
|
+
- the live file `live/<position>.<generation>.ndjson`, `position` the stream's place in
|
|
82
|
+
`streams[]` (from 1). Each line is one trace item as JSON, scrubbed of the run's raw sandbox ids
|
|
83
|
+
as it is appended. A live file is never truncated or rewritten;
|
|
84
|
+
- in run.json, `liveActor` as `humanish.live-actor.v2`: its summary fields (`updatedAt`,
|
|
85
|
+
`persona`, `tokenUsage`, `ids`, `executionProfile`, `providerRequests`, `historyTurnsOmitted`,
|
|
86
|
+
`estimatedCost`) and, in place of `items`, the index of the prefix written so far:
|
|
87
|
+
|
|
88
|
+
```yaml
|
|
89
|
+
liveActor:
|
|
90
|
+
schema: humanish.live-actor.v2
|
|
91
|
+
updatedAt: "<ISO timestamp>"
|
|
92
|
+
itemsFile:
|
|
93
|
+
path: live/1.1.ndjson
|
|
94
|
+
generation: 1
|
|
95
|
+
items: 4210 # lines in the indexed prefix
|
|
96
|
+
bytes: 1048211 # the indexed prefix's size
|
|
97
|
+
sha256: "<hex digest of the indexed prefix>"
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
Each write appends the items past the indexed count, syncs them, then writes run.json, so run.json
|
|
101
|
+
never indexes bytes that are not on disk. The sha256 is a running hash, so no write rehashes a
|
|
102
|
+
file. The writer starts the next generation, holding the participant's full list, when appending
|
|
103
|
+
cannot keep the file equal to that list: the list shrank, an item already written is not the same
|
|
104
|
+
object it wrote, an append failed, or a sandbox id the run learned since occurs in the current
|
|
105
|
+
generation. Once run.json naming the new generation is written, the writer removes every earlier
|
|
106
|
+
one, so at most two generations of a participant exist at once. A failed removal or a failed
|
|
107
|
+
run.json write keeps the earlier generations until the next run.json is written, which removes
|
|
108
|
+
them. The live flush deep-freezes each item it is handed, so a producer that edits an item after
|
|
109
|
+
recording it throws there; a producer records a new item instead.
|
|
110
|
+
|
|
111
|
+
A write that starts a new generation copies the participant's whole list, as every flush did
|
|
112
|
+
before v3. When a participant's generation is replaced on two writes in a row, the run records one
|
|
113
|
+
warn event in `events`, type `live-items.rewritten`, with that stream's `streamId`, and keeps it in
|
|
114
|
+
every later write. A run records at most one.
|
|
115
|
+
|
|
116
|
+
`Run.finish` writes the final trace files and run.json, then removes every live file it wrote and
|
|
117
|
+
`live/`, so a finished v3 bundle holds no live files and differs from v2 only in its schema name.
|
|
118
|
+
A run that ends without its final write (interrupted, or killed) keeps its live files and the
|
|
119
|
+
last run.json that indexes them.
|
|
120
|
+
|
|
121
|
+
Readers load live items through the same loader as traces: `loadStreamTraces` reads exactly the
|
|
122
|
+
indexed bytes of the live file (bytes appended after run.json was written are not read), refuses
|
|
123
|
+
them unless their size, sha256 and line count match the index and each line is a JSON object,
|
|
124
|
+
and returns the stream with `liveActor.items` and without `itemsFile`. It refuses a `liveActor`
|
|
125
|
+
schema it does not know, and inline live items (`humanish.live-actor.v1`) in a v3 run.json. A
|
|
126
|
+
reader that cannot read a live file reads run.json again once and loads that bundle instead, since
|
|
127
|
+
the writer may have removed a superseded generation, or every live file at finish, after the
|
|
128
|
+
reader read run.json. v1 and v2 run.json keep `liveActor.items` inline (`humanish.live-actor.v1`)
|
|
129
|
+
and load as before.
|
|
69
130
|
|
|
70
131
|
Readers that show a run (the Observer server, run detail and the TUI, the review command's
|
|
71
132
|
provenance) load traces through the same loader and rules. A participant whose trace the loader
|
|
@@ -81,10 +142,11 @@ observer-data.json is built from the loaded bundle. From 0.124.0 it is
|
|
|
81
142
|
`humanish.observer-data.v2`: the same data without each participant's trace items, which stay in
|
|
82
143
|
the traces (docs/architecture/observer.md).
|
|
83
144
|
|
|
84
|
-
Compatibility: humanish 0.122.0 and earlier read only v1 and refuse a v2 run.json by its
|
|
85
|
-
(`run schema` fails in verify, analysis refuses the source).
|
|
86
|
-
|
|
87
|
-
|
|
145
|
+
Compatibility: humanish 0.122.0 and earlier read only v1 and refuse a v2 or v3 run.json by its
|
|
146
|
+
schema (`run schema` fails in verify, analysis refuses the source); 0.123.0 and 0.124.0 read v1
|
|
147
|
+
and v2 and refuse v3 the same way. This release reads v1 and v2 bundles as they are, with no
|
|
148
|
+
migration. An exported copy (`humanish export --format bundle`) is written as v3 whatever the
|
|
149
|
+
source version.
|
|
88
150
|
|
|
89
151
|
## Purpose
|
|
90
152
|
|
|
@@ -95,7 +157,7 @@ feedback drafts and future public issues.
|
|
|
95
157
|
## Minimum Bundle Shape
|
|
96
158
|
|
|
97
159
|
```yaml
|
|
98
|
-
schema: humanish.run-bundle.
|
|
160
|
+
schema: humanish.run-bundle.v3 # v2 and v1 bundles are still read
|
|
99
161
|
runId: "<core run id>"
|
|
100
162
|
mode: "dry-run|live"
|
|
101
163
|
simCount: 1
|
|
@@ -270,7 +332,7 @@ An actor trace may include `stopCause` alongside its unchanged `status`,
|
|
|
270
332
|
`completionReason` and verbatim `reason`. Computer-use sessions distinguish
|
|
271
333
|
`provider_output_limit`, `provider_token_limit`, `time_limit`, `spend_limit`,
|
|
272
334
|
`study_spend_limit`, `adapter_limit`, `provider_incomplete`, `provider_status`,
|
|
273
|
-
`provider_refused_prompt`, `harness_aborted`, `usage_unreported`, and `
|
|
335
|
+
`provider_refused_prompt`, `harness_aborted`, `usage_unreported`, `scheduled_end`, and `session_cap`. Absence means the route or
|
|
274
336
|
recording did not retain this precise field; it does not mean the participant
|
|
275
337
|
finished.
|
|
276
338
|
|
|
@@ -392,6 +454,36 @@ contain the older requested-screen `stream.viewport` shape without `desktopGeome
|
|
|
392
454
|
Deterministic Playwright-style adapters remain free to declare a viewport that they also render
|
|
393
455
|
exactly; this hosted-desktop rule does not change that contract.
|
|
394
456
|
|
|
457
|
+
## Desktop Files
|
|
458
|
+
|
|
459
|
+
A hosted computer-use participant's stream carries `desktopFiles` when its study gave it
|
|
460
|
+
`participants[].files`: one entry per file confirmed on the desktop before the participant
|
|
461
|
+
started, in the order they landed.
|
|
462
|
+
|
|
463
|
+
```yaml
|
|
464
|
+
desktopFiles:
|
|
465
|
+
- as: Downloads/insurance-card.png
|
|
466
|
+
size: 251
|
|
467
|
+
sha256: <64 hex characters>
|
|
468
|
+
```
|
|
469
|
+
|
|
470
|
+
`as` is relative to the desktop user's home folder. `size` is the byte count humanish copied,
|
|
471
|
+
which `files.getInfo` on the desktop matched. `sha256` is the digest of those bytes, read on the
|
|
472
|
+
host before any sandbox existed. The contents and the `from` path are not recorded. A participant
|
|
473
|
+
whose copy failed records the files confirmed before the failure. No `desktopFiles` means no file
|
|
474
|
+
was confirmed on that desktop: the study gave the participant none, the run was a dry run, or the
|
|
475
|
+
copy failed before the first file was confirmed. A copy that failed records a `warn` event of type
|
|
476
|
+
`cua-lab.desktop.files.completed`.
|
|
477
|
+
|
|
478
|
+
## Participant Time Zone
|
|
479
|
+
|
|
480
|
+
A computer-use or shared-world participant's stream carries `timeZone`, a tz database name such as
|
|
481
|
+
`Europe/London`, when its study gave it one: `participants[].clock.timeZone`, else
|
|
482
|
+
`actor.clock.timeZone`. Its turns gave the local time in that zone, and on a hosted desktop its
|
|
483
|
+
browser launched with `TZ` set to it. The field records the declared zone, also on a dry run; it is
|
|
484
|
+
not read back from the browser. No `timeZone` means the participant had none, and its hosted
|
|
485
|
+
browser ran in the desktop's UTC.
|
|
486
|
+
|
|
395
487
|
## Subject Provenance
|
|
396
488
|
|
|
397
489
|
`subject` is an optional, additive top-level field: structured provenance for
|
|
@@ -24,38 +24,38 @@ workflow without leaking private upstream truth into core.
|
|
|
24
24
|
|
|
25
25
|
## Contract Index
|
|
26
26
|
|
|
27
|
-
| Contract | Schema
|
|
28
|
-
| --------------------------------- |
|
|
29
|
-
| Run bundle | `humanish.run-bundle.
|
|
30
|
-
| Adapter | `humanish.adapter.v1`
|
|
31
|
-
| Study file | `humanish.study.v3`
|
|
32
|
-
| Persona | `humanish.persona.v1`
|
|
33
|
-
| Scenario | `humanish.scenario.v1`
|
|
34
|
-
| Actor trace | `humanish.actor-trace.v1`
|
|
35
|
-
| Substrate | reserved (never shipped)
|
|
36
|
-
| Evidence stream | reserved (streams live inside the run bundle)
|
|
37
|
-
| Review | `humanish.review.v1`
|
|
38
|
-
| Study analysis | `humanish.study-analysis.v1` (prompt `study-evidence-7` adds `findings[].headline`, `findings[].experience` and `designFindings[]`; older artifacts omit them; `study-evidence-8` adds participant impressions to the instructions, with no field change)
|
|
39
|
-
| Study analysis correction | `humanish.study-analysis-correction.v1`
|
|
40
|
-
| Analysis findings view | `humanish.analysis-findings.v1` (`review --json` field `analysis`; read-only projection of the selected analysis)
|
|
41
|
-
| Analysis execution receipt | `humanish.analysis-execution.v1`
|
|
42
|
-
| Analysis execution start | `humanish.analysis-execution-start.v1`
|
|
43
|
-
| Verification | `humanish.verify-result.v1`
|
|
44
|
-
| Policy | `humanish.policy.v1` (fixture-only; not engine-validated)
|
|
45
|
-
| Feedback | `humanish.feedback.v1`
|
|
46
|
-
| Terminal cost ledger | `humanish.terminal-cost-ledger.v1`
|
|
47
|
-
| Terminal no-spend proof | `humanish.terminal-no-spend-proof.v1`
|
|
48
|
-
| Pricing (operator-editable rates) | `humanish.pricing.v1` (`src/run/pricing.ts`; dated per-model + E2B desktop rates)
|
|
49
|
-
| Run cost summary | `humanish.run-cost-summary.v1` (additive `RunBundle.cost`; estimate, never a charge)
|
|
50
|
-
| Estimated actor cost | `humanish.actor-estimated-cost.v1` (additive `ActorTrace.estimatedCost`)
|
|
51
|
-
| Model settings | `humanish.model-settings.v1` (additive `ActorTrace.modelSettings`; the reasoning effort the request carried)
|
|
52
|
-
| Affordance use | `humanish.affordance-use.v1` (additive `ActorTrace.affordanceUse`; per-class counts of the routes an actor took)
|
|
53
|
-
| Adapter score | `humanish.adapter-score.v1` (`RunBundle.adapterScore`; namespaced; route-specific acceptance semantics)
|
|
54
|
-
| Adapter artifact | `humanish.adapter-artifact.v1` (`RunBundle.adapterArtifacts[]`; namespaced; local relative proof references)
|
|
55
|
-
| Shared-world evidence | `humanish.shared-world.v1` (additive `RunBundle.sharedWorld` + `RunBundle.attributionClass`; `topologyMode: sequential \| concurrent`)
|
|
56
|
-
| Comms thread | `humanish.comms-thread.v1` (off-app email/SMS the app sent, captured; a `kind: log` run-dir artifact of digests only: from/to/subject/link digests + one OTP count, never raw)
|
|
57
|
-
| Serve result | `humanish.serve-result.v1` (`src/observer/serve.ts` is authoritative)
|
|
58
|
-
| Serve control plane | reserved (`/_humanish/api/*` answers `501` `HUMANISH_SERVE_CONTROL_PLANE_DISABLED` in v1)
|
|
27
|
+
| Contract | Schema | Public-safe fixture |
|
|
28
|
+
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------- |
|
|
29
|
+
| Run bundle | `humanish.run-bundle.v3` (each participant's trace in its own file, indexed by `streams[].trace`, and a running participant's items so far in a live file indexed by `streams[].liveActor.itemsFile`; v2 bundles, with running items inline, and v1 bundles, with traces inline, are still read) | `synthetic-run-bundle` |
|
|
30
|
+
| Adapter | `humanish.adapter.v1` | `synthetic-cli-adapter` |
|
|
31
|
+
| Study file | `humanish.study.v3` | `first-run` |
|
|
32
|
+
| Persona | `humanish.persona.v1` | `synthetic-maintainer` |
|
|
33
|
+
| Scenario | `humanish.scenario.v1` | `first-run-smoke` |
|
|
34
|
+
| Actor trace | `humanish.actor-trace.v1` | `synthetic-actor-trace` |
|
|
35
|
+
| Substrate | reserved (never shipped) | none |
|
|
36
|
+
| Evidence stream | reserved (streams live inside the run bundle) | see [`run-bundle.md`](run-bundle.md) |
|
|
37
|
+
| Review | `humanish.review.v1` | `contract-proof-review` |
|
|
38
|
+
| Study analysis | `humanish.study-analysis.v1` (prompt `study-evidence-7` adds `findings[].headline`, `findings[].experience` and `designFindings[]`; older artifacts omit them; `study-evidence-8` adds participant impressions to the instructions, with no field change) | see [study analysis](study-analysis.md) and synthetic analysis fixtures in `tests/` |
|
|
39
|
+
| Study analysis correction | `humanish.study-analysis-correction.v1` | see [study analysis](study-analysis.md#human-review-and-sharing) |
|
|
40
|
+
| Analysis findings view | `humanish.analysis-findings.v1` (`review --json` field `analysis`; read-only projection of the selected analysis) | see [study analysis](study-analysis.md#reading-findings) |
|
|
41
|
+
| Analysis execution receipt | `humanish.analysis-execution.v1` | see [study analysis](study-analysis.md#durable-records) |
|
|
42
|
+
| Analysis execution start | `humanish.analysis-execution-start.v1` | see [study analysis](study-analysis.md#durable-records) |
|
|
43
|
+
| Verification | `humanish.verify-result.v1` | `five-check-verify` |
|
|
44
|
+
| Policy | `humanish.policy.v1` (fixture-only; not engine-validated) | `public-safety-policy` |
|
|
45
|
+
| Feedback | `humanish.feedback.v1` | `public-safe-feedback` |
|
|
46
|
+
| Terminal cost ledger | `humanish.terminal-cost-ledger.v1` | see Terminal Cost Ledger below |
|
|
47
|
+
| Terminal no-spend proof | `humanish.terminal-no-spend-proof.v1` | see Terminal Cost Ledger below |
|
|
48
|
+
| Pricing (operator-editable rates) | `humanish.pricing.v1` (`src/run/pricing.ts`; dated per-model + E2B desktop rates) | see Run Cost Summary And Estimated Actor Cost below |
|
|
49
|
+
| Run cost summary | `humanish.run-cost-summary.v1` (additive `RunBundle.cost`; estimate, never a charge) | see Run Cost Summary And Estimated Actor Cost below |
|
|
50
|
+
| Estimated actor cost | `humanish.actor-estimated-cost.v1` (additive `ActorTrace.estimatedCost`) | see Run Cost Summary And Estimated Actor Cost below |
|
|
51
|
+
| Model settings | `humanish.model-settings.v1` (additive `ActorTrace.modelSettings`; the reasoning effort the request carried) | see Actor Trace below |
|
|
52
|
+
| Affordance use | `humanish.affordance-use.v1` (additive `ActorTrace.affordanceUse`; per-class counts of the routes an actor took) | see Affordance Use below |
|
|
53
|
+
| Adapter score | `humanish.adapter-score.v1` (`RunBundle.adapterScore`; namespaced; route-specific acceptance semantics) | see Product-Adapter Extension Seam below |
|
|
54
|
+
| Adapter artifact | `humanish.adapter-artifact.v1` (`RunBundle.adapterArtifacts[]`; namespaced; local relative proof references) | see Product-Adapter Extension Seam below |
|
|
55
|
+
| Shared-world evidence | `humanish.shared-world.v1` (additive `RunBundle.sharedWorld` + `RunBundle.attributionClass`; `topologyMode: sequential \| concurrent`) | see Shared-World Evidence below |
|
|
56
|
+
| Comms thread | `humanish.comms-thread.v1` (off-app email/SMS the app sent, captured; a `kind: log` run-dir artifact of digests only: from/to/subject/link digests + one OTP count, never raw) | see `comms` under Study File |
|
|
57
|
+
| Serve result | `humanish.serve-result.v1` (`src/observer/serve.ts` is authoritative) | none (command result envelope; see Serve Result below) |
|
|
58
|
+
| Serve control plane | reserved (`/_humanish/api/*` answers `501` `HUMANISH_SERVE_CONTROL_PLANE_DISABLED` in v1) | none |
|
|
59
59
|
|
|
60
60
|
## Study File
|
|
61
61
|
|
|
@@ -210,6 +210,24 @@ The rest of this section names keys by their v3 spelling:
|
|
|
210
210
|
Public/non-loopback targets still require `policies.allowPublicTargets: true`.
|
|
211
211
|
`target` is mutually exclusive with `entry`: `target` is an absolute app-url
|
|
212
212
|
browser target; `entry` is a shared-world participant's same-origin path;
|
|
213
|
+
- `participants[].files` (computer-use on `execution.target: e2b-desktop` only):
|
|
214
|
+
1 to 20 `{ from, as }` entries, each a file copied onto that participant's desktop
|
|
215
|
+
before its browser opens. `from` is relative to the folder that holds the study file
|
|
216
|
+
(`cwd` for a library config); `as` is relative to the desktop user's home folder.
|
|
217
|
+
The parser refuses an absolute path, a `.`, `..` or empty part, a backslash, an `as`
|
|
218
|
+
that starts with `~`, lands in `subject/` or on a name starting with `.humanish`, looks
|
|
219
|
+
like a secret, a local path or encoded text, repeats another `as` or is its folder,
|
|
220
|
+
and a `from` the local-tree denylist names in any case (`isLocalTreeDenylisted` in
|
|
221
|
+
`src/subject/local-tree-archive.ts`). The route also refuses an `as` that holds a
|
|
222
|
+
value from the run's environment, such as a key. The route reads each distinct `from` once, before
|
|
223
|
+
any sandbox and on dry runs too, with `readContainedRegularFile`: a missing file, a
|
|
224
|
+
folder, a symbolic or hard link, a path out of the folder, a file above 10 MiB, more
|
|
225
|
+
than 25 MiB for one participant or 100 MiB of distinct files for the study is
|
|
226
|
+
refused with `HUMANISH_COMPUTER_USE_DESKTOP_FILES_INVALID` (`readDesktopFiles` in
|
|
227
|
+
`src/study/desktop-files.ts`). Each file
|
|
228
|
+
is written with the E2B SDK's `files.write` and its `files.getInfo` size must match, or
|
|
229
|
+
the participant fails before its first turn. The stream records `desktopFiles`
|
|
230
|
+
([run-bundle.md](run-bundle.md#desktop-files)). Other routes refuse the field;
|
|
213
231
|
- a `participants` entry with `count: n` (computer-use E2B route): a group of
|
|
214
232
|
repeated participants. It needs an `id` and the parser expands it to
|
|
215
233
|
`<id>-01` … `<id>-NN`, even when n is 1, before the engine runs (`viewer-01`,
|
|
@@ -235,17 +253,42 @@ The rest of this section names keys by their v3 spelling:
|
|
|
235
253
|
participant's session budget is its window plus 5 minutes (`participantSessionMs` in
|
|
236
254
|
`src/study/arrivals.ts`) in place of `execution.timeoutMs`, and its sandbox deadline adds the
|
|
237
255
|
route's headroom to that. At the end the loop requests the closing report (`debrief.trigger:
|
|
238
|
-
scheduled_end`) and the session ends `goal_satisfied` with `stopCause: scheduled_end`.
|
|
256
|
+
scheduled_end`) and the session ends `goal_satisfied` with `stopCause: scheduled_end`. The
|
|
257
|
+
outcome an accepted report declares, its `outcome` field or the first line of its summary, is
|
|
258
|
+
the trace's `declaredOutcome`. The blocker rule reads a declared `reached` or `blocked` before
|
|
259
|
+
the summary, and after a declared `not_reached` it reads the summary. Before
|
|
239
260
|
the end, a turn that requests no action does not end the session: the trace records a
|
|
240
261
|
`participant ended before its scheduled end` notice, the next request says the time left, and
|
|
241
262
|
the turn counts toward the idle backstop. Before the end, a turn that only waits or takes
|
|
242
263
|
screenshots counts toward neither backstop streak (`advanceBackstop`'s `waitingIsTheTask`). The
|
|
243
264
|
plan entry carries `endAfterMs`, and `study check` prints a `windows` row;
|
|
265
|
+
- `participants[].maxSessionMs` (computer-use only, also on `count:` groups): the longest the
|
|
266
|
+
participant's session lasts, in whole milliseconds from 1 to 86,400,000, counted from the start
|
|
267
|
+
of its own session. `windDownMs` applies to it as to `endAfterMs`. Refused: a value outside that
|
|
268
|
+
range, `maxSessionMs` beside `endAfterMs` on one participant, `maxSessionMs` on `shared-world`,
|
|
269
|
+
and a cap whose sandbox deadline passes the sandbox ceiling. The session budget is the cap plus
|
|
270
|
+
5 minutes (`participantSessionMs`) in place of `execution.timeoutMs`, and the schedule holds the
|
|
271
|
+
participant's slot that long. The route reads the cap's end when the session starts
|
|
272
|
+
(`participantEnd` in `src/study/arrivals.ts`, `participantClock` in
|
|
273
|
+
`src/routes/computer-use/participant-model.ts`) and passes it to the loop as
|
|
274
|
+
`clock.end` with `kind: session_cap`. The participant may end its session before the cap, and
|
|
275
|
+
its waiting turns count toward the backstops. At the cap the loop requests the closing report
|
|
276
|
+
(`debrief.trigger: session_cap`) and the session ends `goal_satisfied` with
|
|
277
|
+
`stopCause: session_cap`, its report's declared outcome read as at a scheduled end. The plan
|
|
278
|
+
entry carries `maxSessionMs`, and the plan line shows
|
|
279
|
+
`cap=<duration>`;
|
|
244
280
|
- `participants[].idleWaitMs` (computer-use and shared-world, also on `count:` groups): this
|
|
245
281
|
participant's `actor.idleWaitMs`, in the same range and refused above an explicit
|
|
246
282
|
`actor.maxWaitMs` (`waitLimitValidationReason`). Its trace records it in `waitSettings`;
|
|
247
|
-
- `actor.clock.timeZone` (computer-use): a tz database time zone name, such as `Europe/London`. Each turn's context hint carries
|
|
248
|
-
the local time in it, and a scheduled end is given in it (UTC without it)
|
|
283
|
+
- `actor.clock.timeZone` (computer-use and shared-world): a tz database time zone name, such as `Europe/London`. Each turn's context hint carries
|
|
284
|
+
the local time in it, and a scheduled end or session cap is given in it (UTC without it). On a hosted desktop the
|
|
285
|
+
participant's browser launches with `TZ` set to it (`openParticipantSurface` in
|
|
286
|
+
`src/routes/computer-use/e2b-desktop/start.ts`). `parseClock` in `src/study/parse/clock.ts` refuses a name Intl does not know, a
|
|
287
|
+
UTC offset and a case variant of Intl's spelling, which Chrome's ICU reads as UTC or `Etc/Unknown`. The computer-use planner
|
|
288
|
+
refuses a zone on the local runtime and the in-process route with `HUMANISH_COMPUTER_USE_TIME_ZONE_UNSUPPORTED`;
|
|
289
|
+
- `participants[].clock.timeZone` (computer-use and shared-world, also on `count:` groups): this participant's `actor.clock`,
|
|
290
|
+
with the same rule. The plan's `limits.timeZone` is the participant's zone, else the actor's. `study check` prints a
|
|
291
|
+
`time zones` row, and each participant's stream in `run.json` records `timeZone`;
|
|
249
292
|
- `execution.concurrency` (computer-use E2B routes, including shared-world): a
|
|
250
293
|
cap on participants in flight at once. When omitted, every participant runs
|
|
251
294
|
simultaneously: both routes resolve it when they plan, from the final participant count
|
|
@@ -293,7 +336,8 @@ The rest of this section names keys by their v3 spelling:
|
|
|
293
336
|
desktop template (image) name or ID the run launches on, for a subject that
|
|
294
337
|
needs runtimes the stock `desktop` image lacks (e.g. node/bun/a local Postgres
|
|
295
338
|
baked into an adopter-maintained image). Any non-empty string is a valid
|
|
296
|
-
name/id (no allowlist); a blank/whitespace value is rejected
|
|
339
|
+
name/id (no allowlist); a blank/whitespace value is rejected, and so is one that
|
|
340
|
+
verify would flag as a secret or local path, as written or encoded. Threaded to the
|
|
297
341
|
SDK's `Sandbox.create(template, opts)` on every desktop-creating route (the
|
|
298
342
|
single-participant and fan-out cua participants, and the shared-world subject and every actor
|
|
299
343
|
sandbox); when absent the call
|
|
@@ -471,7 +515,7 @@ A study's result has one schema on every route, `humanish.study-result.v1`
|
|
|
471
515
|
(`src/run/study-result.ts`), with `route` and `studyId`. 0.109.0 removed `labId`, which 0.108.0
|
|
472
516
|
wrote with the same value. Route-specific fields stay on their route's result. `humanish run`
|
|
473
517
|
with no study, `observe`, and a refusal made before a route is picked keep
|
|
474
|
-
`humanish.run-result.v1`. The evidence record stays `humanish.run-bundle.
|
|
518
|
+
`humanish.run-result.v1`. The evidence record stays `humanish.run-bundle.v3` in every case.
|
|
475
519
|
The computer-use result carries `plan` (the pre-flight participant table: concurrency, waves,
|
|
476
520
|
per-participant session budget, worst-case sandbox-minutes), `lanes[]` (always present,
|
|
477
521
|
length 1 at N=1; per-participant status/session/sandbox/subject), and `laneSummary`
|
|
@@ -616,7 +660,7 @@ Adapter-owned fields:
|
|
|
616
660
|
Abbreviated synthetic fixture (omits `cwd`, `simulations`, `streams` and `events`):
|
|
617
661
|
|
|
618
662
|
```yaml
|
|
619
|
-
schema: humanish.run-bundle.
|
|
663
|
+
schema: humanish.run-bundle.v3
|
|
620
664
|
runId: synthetic-run-bundle-2026-06-02t10-00-00-000z-proof
|
|
621
665
|
mode: dry-run
|
|
622
666
|
simCount: 1
|
|
@@ -967,7 +1011,7 @@ output all map onto one `ActorTrace`. Registered actors live in
|
|
|
967
1011
|
`pi-agent-core` or `claude-agent-sdk` fails to parse. There is no
|
|
968
1012
|
`humanish.actor.v1`; that name never shipped.
|
|
969
1013
|
|
|
970
|
-
Where the trace is stored depends on the run bundle version.
|
|
1014
|
+
Where the trace is stored depends on the run bundle version. From `humanish.run-bundle.v2`,
|
|
971
1015
|
`streams[].actor` in run.json is the trace without `items`, and the full trace, with the stream's
|
|
972
1016
|
screenshot entries as `artifacts`, is the file `streams[].trace.path` names (`actor.json`,
|
|
973
1017
|
`actors/<stream>.json` or `actor-<surface>.json`). A v1 run.json holds the full trace inline.
|
|
@@ -1010,6 +1054,32 @@ Core-owned fields:
|
|
|
1010
1054
|
click-like `ui_action` items, structured `coord` (`x`/`y`). Both are additive:
|
|
1011
1055
|
absent on older bundles and non-stamping producers, and absence means
|
|
1012
1056
|
"timing/position unrecorded", never zero
|
|
1057
|
+
- optional `items[].page` and `finalPage` (`{ origin, path }`, `TracePage` in
|
|
1058
|
+
`src/actors/contract.ts`): where a computer-use participant's browser was. The loop puts `page`
|
|
1059
|
+
on the `screenshot` item of each observation that read a URL: `turn-00-start` is the page before
|
|
1060
|
+
any action, `turn-01` the page after turn 1's actions, and dwell frames have their own.
|
|
1061
|
+
`finalPage` is the page of the session's last observation, which is a closing observation with
|
|
1062
|
+
no item when tasks are declared; it is absent when that observation read no URL. The screenshot
|
|
1063
|
+
items with a `page`, in item order, then `finalPage`, list the participant's path through the
|
|
1064
|
+
app. `origin` is the URL's scheme, host and port; `path` is its percent-encoded path. A query
|
|
1065
|
+
string, fragment and user info are never kept, and a non-web URL keeps its scheme only
|
|
1066
|
+
(`{ origin: "about:", path: "" }`). Redaction (`tracePage` in `src/evidence/trace-page.ts`):
|
|
1067
|
+
a path segment becomes `[REDACTED_URL_PART]` when it percent-encodes a letter, digit, `-`, `.`,
|
|
1068
|
+
`_`, `~` or `%`, or when any decoded reading of it is email-shaped, holds a run of 16 or more letters,
|
|
1069
|
+
digits, `-` and `_` with a piece of 8 or more characters mixing letters and digits, or changes
|
|
1070
|
+
under the trace text redaction (the study's secrets, then `redactText`). The whole path then
|
|
1071
|
+
goes through that redaction, is cut to 1024 characters ending `[TRUNCATED]`, and is withheld
|
|
1072
|
+
whole when verify's trace scan would still read it as sensitive or opaque. A host label
|
|
1073
|
+
shaped like that token run becomes `[REDACTED_URL_PART]`, then the origin goes through the same
|
|
1074
|
+
redaction (an E2B sandbox host becomes `[REDACTED_SECRET]`). When origin and path together
|
|
1075
|
+
still change under that redaction (a study secret that is a whole link), both become
|
|
1076
|
+
`[REDACTED_URL_PART]`. A token shorter than 16 characters, an
|
|
1077
|
+
all-digit id and personal data in readable words stay, so the sharing rules for synthetic data
|
|
1078
|
+
still apply. Pages come from the browser-state probe, which hosted runs have on Chrome and
|
|
1079
|
+
Chromium only; a Firefox participant, or one whose probe is unavailable, has screenshots without
|
|
1080
|
+
pages. The local browser route and the shared-world route record none. verify reports a page
|
|
1081
|
+
whose path holds `?` or `#`, or whose origin is neither a redaction marker nor a scheme with an
|
|
1082
|
+
optional host and port, as a public-safety finding. Both fields are additive; absence means "page not recorded"
|
|
1013
1083
|
- optional `modelSettings` (`humanish.model-settings.v1`): how the model was
|
|
1014
1084
|
asked to run, alongside `ids.model` which says which model it was.
|
|
1015
1085
|
`reasoningEffort` records the value the request actually carried,
|
|
@@ -1829,7 +1899,7 @@ including existence for referenced adapter artifacts.
|
|
|
1829
1899
|
|
|
1830
1900
|
Reserved: `humanish.evidence-stream.v1` has never shipped as a standalone
|
|
1831
1901
|
schema, and streams are not standalone artifacts. They are the `streams` array
|
|
1832
|
-
inside the run bundle (`humanish.run-bundle.
|
|
1902
|
+
inside the run bundle (`humanish.run-bundle.v3`), normalizing UI, browser, terminal, TUI,
|
|
1833
1903
|
code-agent UI, artifact, and summary streams, each with transport, terminal
|
|
1834
1904
|
tail, completion, meaningful-use verdicts, and artifact pointers. See
|
|
1835
1905
|
[`run-bundle.md`](run-bundle.md#completion-and-meaningful-use-verdicts) for
|