humanish 0.108.0 → 0.109.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ARCHITECTURE.md +58 -59
- package/CHANGELOG.md +22 -0
- package/CONTEXT.md +44 -44
- package/README.md +3 -3
- package/TELEMETRY.md +2 -2
- package/dist/actors/codex/app-server-trace.js +2 -1
- package/dist/actors/codex/restricted-participant.js +12 -12
- package/dist/actors/codex/restricted-session.d.ts +2 -2
- package/dist/actors/computer-use/actor.d.ts +6 -6
- package/dist/actors/computer-use/actor.js +2 -2
- package/dist/actors/computer-use/admission-limit.d.ts +2 -2
- package/dist/actors/computer-use/admission-limit.js +3 -3
- package/dist/actors/computer-use/executor-error.d.ts +2 -2
- package/dist/actors/computer-use/executor-error.js +3 -3
- package/dist/actors/computer-use/loop/actions.js +5 -5
- package/dist/actors/computer-use/loop/debrief.js +2 -2
- package/dist/actors/computer-use/loop/ending.d.ts +1 -1
- package/dist/actors/computer-use/loop/ending.js +12 -11
- package/dist/actors/computer-use/loop/observation.js +6 -5
- package/dist/actors/computer-use/loop/provider-call.js +11 -11
- package/dist/actors/computer-use/loop/session.d.ts +4 -4
- package/dist/actors/computer-use/loop/session.js +2 -2
- package/dist/actors/computer-use/loop/trace.js +3 -2
- package/dist/actors/computer-use/loop/types.d.ts +24 -24
- package/dist/actors/computer-use/loop/usage.d.ts +2 -2
- package/dist/actors/computer-use/loop.d.ts +6 -1
- package/dist/actors/computer-use/loop.js +26 -4
- package/dist/actors/computer-use/openai-provider.d.ts +2 -2
- package/dist/actors/computer-use/openai-provider.js +8 -8
- package/dist/actors/computer-use/provider-error.d.ts +4 -4
- package/dist/actors/computer-use/provider-error.js +6 -6
- package/dist/actors/contract.d.ts +4 -4
- package/dist/actors/contract.js +1 -1
- package/dist/actors/local-agent/claude-session.js +3 -3
- package/dist/actors/local-agent/cli.d.ts +1 -1
- package/dist/actors/local-agent/cli.js +1 -1
- package/dist/actors/reasoning-effort.js +3 -3
- package/dist/actors/registry.d.ts +2 -2
- package/dist/actors/registry.js +3 -3
- package/dist/actors/scripted-browser/actor.d.ts +2 -2
- package/dist/actors/scripted-browser/actor.js +1 -1
- package/dist/analysis/automatic-completion.d.ts +7 -7
- package/dist/analysis/automatic-completion.js +1 -1
- package/dist/analysis/automatic-config.d.ts +6 -6
- package/dist/analysis/automatic-config.js +1 -1
- package/dist/analysis/execute.d.ts +1 -1
- package/dist/analysis/execute.js +1 -1
- package/dist/analysis/types.d.ts +6 -0
- package/dist/browser-control/client.js +16 -16
- package/dist/browser-control/dispatcher.js +4 -4
- package/dist/browser-control/protocol.js +14 -14
- package/dist/browser-control/recording-transfer.js +4 -4
- package/dist/browser-control/transport.js +7 -7
- package/dist/cli/commands/analysis-signals.d.ts +2 -2
- package/dist/cli/commands/comms.js +14 -22
- package/dist/cli/commands/migrate.js +15 -4
- package/dist/cli/commands/observe.d.ts +0 -1
- package/dist/cli/commands/observe.js +0 -15
- package/dist/cli/commands/run-command.d.ts +2 -2
- package/dist/cli/commands/run-command.js +3 -3
- package/dist/cli/commands/runs.js +9 -7
- package/dist/cli/commands/setup.js +5 -7
- package/dist/cli/commands/study-format.d.ts +9 -0
- package/dist/cli/commands/{lab-format.js → study-format.js} +5 -5
- package/dist/cli/commands/{lab-route-computer-use.d.ts → study-route-computer-use.d.ts} +6 -6
- package/dist/cli/commands/{lab-route-computer-use.js → study-route-computer-use.js} +12 -12
- package/dist/cli/commands/{lab-route-open.d.ts → study-route-open.d.ts} +4 -4
- package/dist/cli/commands/{lab-route-open.js → study-route-open.js} +2 -2
- package/dist/cli/commands/{lab-route-preview.d.ts → study-route-preview.d.ts} +7 -7
- package/dist/cli/commands/{lab-route-preview.js → study-route-preview.js} +3 -3
- package/dist/cli/commands/study-route-run.d.ts +19 -0
- package/dist/cli/commands/{lab-route-run.js → study-route-run.js} +6 -6
- package/dist/cli/commands/study-route-scripted.d.ts +17 -0
- package/dist/cli/commands/{lab-route-scripted.js → study-route-scripted.js} +4 -4
- package/dist/cli/commands/{lab-route-shared-world.d.ts → study-route-shared-world.d.ts} +5 -5
- package/dist/cli/commands/{lab-route-shared-world.js → study-route-shared-world.js} +7 -7
- package/dist/cli/commands/study-route-terminal.d.ts +17 -0
- package/dist/cli/commands/{lab-route-terminal.js → study-route-terminal.js} +4 -4
- package/dist/cli/commands/study-run.d.ts +9 -0
- package/dist/cli/commands/{lab-run.js → study-run.js} +34 -34
- package/dist/cli/commands/{lab-scorer.d.ts → study-scorer.d.ts} +5 -5
- package/dist/cli/commands/{lab-scorer.js → study-scorer.js} +1 -1
- package/dist/cli/commands/study.js +13 -49
- package/dist/cli/commands/tui.js +12 -12
- package/dist/cli/commands/watch.js +30 -65
- package/dist/cli/deprecations.d.ts +1 -13
- package/dist/cli/deprecations.js +0 -30
- package/dist/cli/doctor.d.ts +3 -3
- package/dist/cli/doctor.js +16 -16
- package/dist/cli/first-run-path.d.ts +4 -4
- package/dist/cli/first-run-path.js +7 -7
- package/dist/cli/io.d.ts +3 -3
- package/dist/cli/io.js +2 -2
- package/dist/cli/observer-follow.d.ts +2 -2
- package/dist/cli/observer-follow.js +2 -2
- package/dist/cli/orientation.d.ts +4 -4
- package/dist/cli/orientation.js +58 -20
- package/dist/cli/program.js +3 -4
- package/dist/cli/telemetry.d.ts +1 -1
- package/dist/cli/telemetry.js +14 -20
- package/dist/comms/catch-host.d.ts +1 -1
- package/dist/comms/catch-host.js +3 -3
- package/dist/comms/connections.d.ts +1 -1
- package/dist/comms/connections.js +2 -2
- package/dist/comms/external-evidence.d.ts +5 -5
- package/dist/comms/external-evidence.js +4 -3
- package/dist/comms/fake-inbox.d.ts +2 -2
- package/dist/comms/fake-inbox.js +2 -2
- package/dist/comms/receiving-runtime.d.ts +4 -4
- package/dist/comms/sandbox-catch.d.ts +2 -2
- package/dist/comms/sandbox-catch.js +2 -2
- package/dist/comms/setup.d.ts +2 -2
- package/dist/comms/setup.js +29 -32
- package/dist/evidence/redaction.d.ts +11 -0
- package/dist/evidence/redaction.js +43 -0
- package/dist/feedback/draft.js +2 -2
- package/dist/feedback/export-bundle.js +5 -4
- package/dist/feedback/export.js +3 -2
- package/dist/feedback/proof.d.ts +1 -1
- package/dist/feedback/proof.js +2 -2
- package/dist/guest/bootstrap.js +6 -6
- package/dist/guest/browser-tools.js +8 -8
- package/dist/guest/chromium-text.js +17 -17
- package/dist/guest/desktop-executor.d.ts +2 -2
- package/dist/guest/desktop-executor.js +20 -20
- package/dist/guest/desktop-keys.js +4 -4
- package/dist/guest/desktop-media.d.ts +2 -2
- package/dist/guest/desktop-media.js +16 -16
- package/dist/guest/desktop-native.js +10 -10
- package/dist/guest/runtime-desktop.js +2 -2
- package/dist/guest/runtime-processes.d.ts +2 -2
- package/dist/guest/runtime-processes.js +2 -2
- package/dist/guest/runtime.js +5 -5
- package/dist/index.d.ts +13 -15
- package/dist/index.js +6 -8
- package/dist/observer/artifact.d.ts +1 -1
- package/dist/observer/artifact.js +1 -1
- package/dist/observer/data.d.ts +1 -1
- package/dist/observer/library.d.ts +2 -0
- package/dist/observer/library.js +4 -1
- package/dist/observer/render.js +7 -1
- package/dist/observer/run-routes.d.ts +1 -0
- package/dist/observer/run-routes.js +37 -12
- package/dist/observer/serve.d.ts +1 -1
- package/dist/observer-app.html +1 -1
- package/dist/routes/computer-use/bundle-parts.js +3 -2
- package/dist/routes/computer-use/bundle.d.ts +2 -2
- package/dist/routes/computer-use/bundle.js +7 -6
- package/dist/routes/computer-use/e2b-desktop/comms.d.ts +5 -5
- package/dist/routes/computer-use/e2b-desktop/comms.js +5 -4
- package/dist/routes/computer-use/e2b-desktop/fidelity.d.ts +1 -1
- package/dist/routes/computer-use/e2b-desktop/prepare.d.ts +1 -1
- package/dist/routes/computer-use/e2b-desktop/prepare.js +3 -3
- package/dist/routes/computer-use/e2b-desktop/start.js +3 -4
- package/dist/routes/computer-use/e2b-desktop/state.d.ts +2 -2
- package/dist/routes/computer-use/e2b-desktop/teardown.js +2 -3
- package/dist/routes/computer-use/external-comms.d.ts +3 -3
- package/dist/routes/computer-use/fanout-bundle.js +11 -10
- package/dist/routes/computer-use/fanout-records.js +2 -2
- package/dist/routes/computer-use/in-process-desktop.js +1 -1
- package/dist/routes/computer-use/live-flush.d.ts +3 -3
- package/dist/routes/computer-use/live-phase.d.ts +2 -3
- package/dist/routes/computer-use/live-phase.js +3 -3
- package/dist/routes/computer-use/local-tree-pack.d.ts +5 -5
- package/dist/routes/computer-use/local-tree-pack.js +1 -1
- package/dist/routes/computer-use/local-vm.d.ts +8 -8
- package/dist/routes/computer-use/local-vm.js +2 -2
- package/dist/routes/computer-use/participant-desktop.d.ts +3 -3
- package/dist/routes/computer-use/participant-desktop.js +1 -1
- package/dist/routes/computer-use/participant-execution.js +1 -1
- package/dist/routes/computer-use/participant-model.js +3 -3
- package/dist/routes/computer-use/participant-prompt.d.ts +5 -5
- package/dist/routes/computer-use/participant-prompt.js +3 -3
- package/dist/routes/computer-use/participant-runs.d.ts +10 -10
- package/dist/routes/computer-use/participant-runs.js +14 -13
- package/dist/routes/computer-use/plan.d.ts +12 -12
- package/dist/routes/computer-use/plan.js +27 -27
- package/dist/routes/computer-use/preflight.d.ts +5 -5
- package/dist/routes/computer-use/preflight.js +5 -5
- package/dist/routes/computer-use/rerun-selection.d.ts +1 -1
- package/dist/routes/computer-use/rerun-selection.js +3 -2
- package/dist/routes/computer-use/result.d.ts +4 -4
- package/dist/routes/computer-use/result.js +13 -12
- package/dist/routes/computer-use/route.d.ts +12 -12
- package/dist/routes/computer-use/route.js +19 -19
- package/dist/routes/computer-use/self-report.js +1 -1
- package/dist/routes/computer-use/setup.d.ts +22 -22
- package/dist/routes/computer-use/setup.js +15 -15
- package/dist/routes/computer-use/single-bundle.d.ts +4 -4
- package/dist/routes/computer-use/single-bundle.js +14 -13
- package/dist/routes/computer-use/subject-projection.d.ts +3 -3
- package/dist/routes/computer-use/types.d.ts +34 -34
- package/dist/routes/computer-use/types.js +3 -3
- package/dist/routes/preview.d.ts +6 -6
- package/dist/routes/preview.js +5 -5
- package/dist/routes/scripted/bundle.d.ts +6 -6
- package/dist/routes/scripted/bundle.js +10 -9
- package/dist/routes/scripted/plan.d.ts +7 -7
- package/dist/routes/scripted/plan.js +6 -6
- package/dist/routes/scripted/result.d.ts +3 -3
- package/dist/routes/scripted/result.js +9 -9
- package/dist/routes/scripted/route.d.ts +9 -9
- package/dist/routes/scripted/route.js +11 -11
- package/dist/routes/scripted/scenario.d.ts +1 -1
- package/dist/routes/scripted/scenario.js +3 -3
- package/dist/routes/scripted/setup.d.ts +6 -6
- package/dist/routes/scripted/setup.js +4 -4
- package/dist/routes/scripted/subject.d.ts +5 -5
- package/dist/routes/scripted/subject.js +3 -3
- package/dist/routes/scripted/surface-records.d.ts +1 -1
- package/dist/routes/scripted/surface-records.js +2 -2
- package/dist/routes/scripted/surface-sessions.d.ts +3 -3
- package/dist/routes/scripted/types.d.ts +17 -17
- package/dist/routes/scripted/types.js +1 -1
- package/dist/routes/shared-world/bundle-records.js +3 -3
- package/dist/routes/shared-world/bundle.js +15 -14
- package/dist/routes/shared-world/checkpoints.d.ts +3 -3
- package/dist/routes/shared-world/checkpoints.js +1 -1
- package/dist/routes/shared-world/comms.d.ts +9 -9
- package/dist/routes/shared-world/comms.js +6 -5
- package/dist/routes/shared-world/participant-specs.d.ts +3 -3
- package/dist/routes/shared-world/participant-specs.js +7 -7
- package/dist/routes/shared-world/plan.d.ts +9 -9
- package/dist/routes/shared-world/plan.js +9 -9
- package/dist/routes/shared-world/provisioned.d.ts +8 -8
- package/dist/routes/shared-world/provisioned.js +4 -4
- package/dist/routes/shared-world/result.d.ts +6 -6
- package/dist/routes/shared-world/result.js +11 -10
- package/dist/routes/shared-world/route.d.ts +11 -11
- package/dist/routes/shared-world/route.js +19 -19
- package/dist/routes/shared-world/setup.d.ts +13 -13
- package/dist/routes/shared-world/setup.js +19 -19
- package/dist/routes/shared-world/types.d.ts +27 -27
- package/dist/routes/terminal/adapter.d.ts +2 -2
- package/dist/routes/terminal/adapter.js +4 -4
- package/dist/routes/terminal/bundle.d.ts +11 -11
- package/dist/routes/terminal/bundle.js +12 -12
- package/dist/routes/terminal/credentials.d.ts +4 -4
- package/dist/routes/terminal/credentials.js +5 -5
- package/dist/routes/terminal/dry-run.d.ts +4 -4
- package/dist/routes/terminal/dry-run.js +9 -9
- package/dist/routes/terminal/ledger.d.ts +2 -2
- package/dist/routes/terminal/lifetime.d.ts +1 -1
- package/dist/routes/terminal/lifetime.js +1 -1
- package/dist/routes/terminal/live-finish.d.ts +2 -2
- package/dist/routes/terminal/live-finish.js +7 -7
- package/dist/routes/terminal/live-sandbox.d.ts +2 -2
- package/dist/routes/terminal/live-sandbox.js +3 -2
- package/dist/routes/terminal/participant-text.js +2 -1
- package/dist/routes/terminal/persona.d.ts +2 -2
- package/dist/routes/terminal/persona.js +2 -2
- package/dist/routes/terminal/plan.d.ts +7 -7
- package/dist/routes/terminal/plan.js +5 -5
- package/dist/routes/terminal/result.d.ts +5 -5
- package/dist/routes/terminal/result.js +7 -7
- package/dist/routes/terminal/route.d.ts +9 -9
- package/dist/routes/terminal/route.js +14 -14
- package/dist/routes/terminal/session.d.ts +5 -5
- package/dist/routes/terminal/session.js +7 -7
- package/dist/routes/terminal/trace.d.ts +2 -2
- package/dist/routes/terminal/types.d.ts +24 -24
- package/dist/routes/terminal/types.js +2 -2
- package/dist/run/bundle.d.ts +7 -7
- package/dist/run/bundle.js +2 -2
- package/dist/run/costs.d.ts +23 -0
- package/dist/run/costs.js +72 -43
- package/dist/run/dry-run-inputs.d.ts +1 -1
- package/dist/run/dry-run-inputs.js +1 -1
- package/dist/run/dry-run.js +2 -2
- package/dist/run/outcomes.d.ts +1 -1
- package/dist/run/participant.d.ts +1 -1
- package/dist/run/preflight-receipts.js +1 -1
- package/dist/run/pricing.js +2 -2
- package/dist/run/projection.d.ts +32 -32
- package/dist/run/projection.js +44 -44
- package/dist/run/reclaim.d.ts +1 -1
- package/dist/run/reclaim.js +1 -1
- package/dist/run/results.d.ts +2 -2
- package/dist/run/run-cost.d.ts +53 -0
- package/dist/run/run-cost.js +105 -0
- package/dist/run/run-index-analysis.d.ts +15 -0
- package/dist/run/run-index-analysis.js +77 -0
- package/dist/run/run-index.d.ts +11 -1
- package/dist/run/run-index.js +63 -49
- package/dist/run/run.d.ts +3 -3
- package/dist/run/run.js +2 -2
- package/dist/run/shared-world-evidence.d.ts +1 -1
- package/dist/run/shared-world-shape.js +1 -1
- package/dist/run/stats.d.ts +3 -3
- package/dist/run/stats.js +14 -14
- package/dist/run/status.d.ts +10 -2
- package/dist/run/status.js +6 -3
- package/dist/run/streams.d.ts +2 -2
- package/dist/run/study-provenance.d.ts +4 -2
- package/dist/run/study-provenance.js +7 -4
- package/dist/run/study-result.d.ts +0 -5
- package/dist/run/study-result.js +1 -1
- package/dist/run-study.d.ts +112 -0
- package/dist/{run-lab.js → run-study.js} +52 -45
- package/dist/{lab → study}/adapter-extension.d.ts +8 -20
- package/dist/{lab → study}/adapter-extension.js +2 -14
- package/dist/{lab → study}/adapter-scorer-loader.d.ts +10 -10
- package/dist/{lab → study}/adapter-scorer-loader.js +3 -3
- package/dist/study/composition-rules.d.ts +7 -0
- package/dist/{lab → study}/composition-rules.js +12 -12
- package/dist/study/config.d.ts +15 -0
- package/dist/{lab → study}/config.js +37 -20
- package/dist/study/convert.d.ts +1 -1
- package/dist/study/convert.js +15 -15
- package/dist/{lab → study}/device-presets.d.ts +2 -2
- package/dist/{lab → study}/discover.d.ts +29 -23
- package/dist/{lab → study}/discover.js +97 -118
- package/dist/{lab → study}/doctor.d.ts +11 -11
- package/dist/{lab → study}/doctor.js +18 -18
- package/dist/study/files.d.ts +2 -2
- package/dist/study/files.js +6 -5
- package/dist/{lab → study}/init-templates.d.ts +2 -2
- package/dist/{lab → study}/init-templates.js +16 -11
- package/dist/{lab → study}/init.d.ts +1 -1
- package/dist/{lab → study}/init.js +7 -4
- package/dist/{lab → study}/keys.d.ts +2 -2
- package/dist/{lab → study}/keys.js +10 -10
- package/dist/study/migrate.d.ts +8 -3
- package/dist/study/migrate.js +60 -4
- package/dist/{lab → study}/parse/actors.d.ts +5 -5
- package/dist/{lab → study}/parse/actors.js +5 -5
- package/dist/study/parse/comms.d.ts +12 -0
- package/dist/study/parse/execution.d.ts +19 -0
- package/dist/{lab → study}/parse/study-v3.d.ts +3 -5
- package/dist/{lab → study}/parse/study-v3.js +3 -3
- package/dist/{lab → study}/parse/subject-state.d.ts +8 -8
- package/dist/{lab → study}/parse/subject-state.js +4 -4
- package/dist/{lab → study}/parse/subject.d.ts +3 -3
- package/dist/{lab → study}/parse/subject.js +8 -8
- package/dist/{lab → study}/parse/values.d.ts +2 -2
- package/dist/{lab → study}/persona-resolve.d.ts +4 -4
- package/dist/{lab → study}/persona-resolve.js +2 -2
- package/dist/{lab → study}/plan-base.d.ts +7 -7
- package/dist/{lab → study}/plan-base.js +1 -1
- package/dist/{lab → study}/plan-participants.d.ts +10 -10
- package/dist/{lab → study}/plan-participants.js +4 -4
- package/dist/{lab → study}/plan-types.d.ts +28 -28
- package/dist/{lab → study}/plan-types.js +2 -2
- package/dist/study/plan.d.ts +13 -0
- package/dist/{lab → study}/plan.js +20 -20
- package/dist/study/preflight-probes.d.ts +3 -0
- package/dist/{lab → study}/preflight-probes.js +9 -8
- package/dist/study/preflight-result.d.ts +8 -0
- package/dist/{lab → study}/preflight-result.js +3 -3
- package/dist/{lab → study}/preflight.d.ts +31 -31
- package/dist/{lab → study}/preflight.js +8 -8
- package/dist/{lab → study}/requirements.d.ts +3 -3
- package/dist/{lab → study}/requirements.js +1 -1
- package/dist/study/route-inputs.d.ts +28 -0
- package/dist/{lab → study}/route-inputs.js +1 -1
- package/dist/{lab → study}/routing.d.ts +12 -12
- package/dist/{lab/run-lab-events.d.ts → study/run-study-events.d.ts} +10 -10
- package/dist/{lab/run-lab-events.js → study/run-study-events.js} +4 -4
- package/dist/{lab/run-lab-homes.d.ts → study/run-study-homes.d.ts} +8 -17
- package/dist/study/run-study-homes.js +3 -0
- package/dist/study/run-study-options.d.ts +28 -0
- package/dist/{lab/run-lab-options.js → study/run-study-options.js} +16 -15
- package/dist/{lab/lab-deps.d.ts → study/study-deps.d.ts} +2 -2
- package/dist/{lab/lab-deps.js → study/study-deps.js} +2 -2
- package/dist/{lab → study}/summary.d.ts +10 -10
- package/dist/{lab → study}/summary.js +15 -15
- package/dist/{lab → study}/tasks.d.ts +3 -3
- package/dist/{lab → study}/tasks.js +1 -1
- package/dist/{lab → study}/types.d.ts +104 -104
- package/dist/{lab → study}/types.js +1 -1
- package/dist/{lab → study}/validation.d.ts +19 -19
- package/dist/{lab → study}/validation.js +13 -13
- package/dist/{lab → study}/warnings.d.ts +4 -4
- package/dist/{lab → study}/warnings.js +1 -1
- package/dist/subject/clone.d.ts +6 -6
- package/dist/subject/clone.js +2 -2
- package/dist/subject/local-tree-archive.d.ts +2 -2
- package/dist/subject/local-tree-archive.js +3 -3
- package/dist/subject/local-tree.d.ts +3 -3
- package/dist/subject/runtime.js +2 -2
- package/dist/subject/serve.d.ts +4 -4
- package/dist/subject/state.d.ts +3 -3
- package/dist/subject/state.js +1 -1
- package/dist/subject/steps.d.ts +1 -1
- package/dist/subject/steps.js +1 -1
- package/dist/substrates/desktop-session.js +2 -2
- package/dist/substrates/detached.d.ts +1 -1
- package/dist/substrates/e2b/desktop-browser.d.ts +4 -4
- package/dist/substrates/e2b/desktop-browser.js +1 -1
- package/dist/substrates/e2b/desktop-cdp.d.ts +1 -1
- package/dist/substrates/e2b/desktop-cdp.js +1 -1
- package/dist/substrates/e2b/desktop-executor.d.ts +1 -1
- package/dist/substrates/e2b/desktop-executor.js +12 -12
- package/dist/substrates/e2b/desktop-geometry.d.ts +1 -1
- package/dist/substrates/e2b/desktop-geometry.js +1 -1
- package/dist/substrates/e2b/desktop-media.d.ts +3 -3
- package/dist/substrates/e2b/lifetime.js +1 -1
- package/dist/substrates/e2b/sandbox.d.ts +3 -3
- package/dist/substrates/e2b/sandbox.js +1 -1
- package/dist/substrates/e2b/sdk.d.ts +1 -1
- package/dist/substrates/e2b/sdk.js +1 -1
- package/dist/substrates/local/runtime-config.d.ts +4 -4
- package/dist/substrates/local/runtime-config.js +4 -4
- package/dist/tui/contract.d.ts +8 -8
- package/dist/tui/contract.js +2 -2
- package/dist/tui/launch.d.ts +3 -3
- package/dist/tui/launch.js +22 -16
- package/dist/tui-app.js +139 -139
- package/dist/verify/subject.d.ts +1 -1
- package/dist/verify/subject.js +1 -1
- package/dist/verify/verify.js +2 -1
- package/docs/architecture/actor-contract.md +5 -5
- package/docs/architecture/browser-control.md +1 -1
- package/docs/architecture/external-public-shared-world.md +8 -7
- package/docs/architecture/github-feedback-loop.md +1 -1
- package/docs/architecture/task-protocol-support.md +1 -1
- package/docs/architecture/terminal-product-route.md +18 -18
- package/docs/contracts/adapter-fixtures.md +2 -2
- package/docs/contracts/errors.md +8 -8
- package/docs/contracts/run-bundle.md +6 -4
- package/docs/contracts/schemas.md +61 -42
- package/docs/contracts/study-analysis.md +4 -1
- package/docs/contracts/study-costs.md +17 -2
- package/examples/README.md +1 -1
- package/examples/scorer/scorer.mjs +3 -3
- package/package.json +2 -1
- package/skills/humanish/SKILL.md +8 -6
- package/skills/humanish/agents/openai.yaml +1 -1
- package/dist/cli/commands/lab-format.d.ts +0 -9
- package/dist/cli/commands/lab-route-run.d.ts +0 -19
- package/dist/cli/commands/lab-route-scripted.d.ts +0 -17
- package/dist/cli/commands/lab-route-terminal.d.ts +0 -17
- package/dist/cli/commands/lab-run.d.ts +0 -9
- package/dist/lab/composition-rules.d.ts +0 -7
- package/dist/lab/config.d.ts +0 -7
- package/dist/lab/deprecated-fields.d.ts +0 -10
- package/dist/lab/deprecated-fields.js +0 -24
- package/dist/lab/parse/comms.d.ts +0 -12
- package/dist/lab/parse/execution.d.ts +0 -19
- package/dist/lab/plan.d.ts +0 -13
- package/dist/lab/preflight-probes.d.ts +0 -3
- package/dist/lab/preflight-result.d.ts +0 -8
- package/dist/lab/route-inputs.d.ts +0 -28
- package/dist/lab/run-lab-homes.js +0 -7
- package/dist/lab/run-lab-options.d.ts +0 -28
- package/dist/library-aliases.d.ts +0 -56
- package/dist/library-aliases.js +0 -22
- package/dist/run-lab.d.ts +0 -112
- /package/dist/{lab → study}/device-presets.js +0 -0
- /package/dist/{lab → study}/init-paths.d.ts +0 -0
- /package/dist/{lab → study}/init-paths.js +0 -0
- /package/dist/{lab → study}/init-plan.d.ts +0 -0
- /package/dist/{lab → study}/init-plan.js +0 -0
- /package/dist/{lab → study}/parse/comms.js +0 -0
- /package/dist/{lab → study}/parse/execution.js +0 -0
- /package/dist/{lab → study}/parse/values.js +0 -0
- /package/dist/{lab → study}/participant-assignment.d.ts +0 -0
- /package/dist/{lab → study}/participant-assignment.js +0 -0
- /package/dist/{lab → study}/persona.d.ts +0 -0
- /package/dist/{lab → study}/persona.js +0 -0
- /package/dist/{lab → study}/routing.js +0 -0
package/dist/verify/subject.d.ts
CHANGED
|
@@ -9,7 +9,7 @@ import type { RunBundle } from "../run/bundle.js";
|
|
|
9
9
|
export declare function subjectStateFindings(bundle: RunBundle): string[];
|
|
10
10
|
/**
|
|
11
11
|
* Advisory (never flips ok): a live clone bundle whose subject env is provisioned while its
|
|
12
|
-
* state story is undeclared probably points at state the
|
|
12
|
+
* state story is undeclared probably points at state the study does not control. Emitted at
|
|
13
13
|
* most once per bundle (the subject block is bundle-level, never per stream). GITHUB_TOKEN
|
|
14
14
|
* is mechanically excluded: the harness consumes that name for clone auth, and it carries no
|
|
15
15
|
* state implication.
|
package/dist/verify/subject.js
CHANGED
|
@@ -100,7 +100,7 @@ export function subjectStateFindings(bundle) {
|
|
|
100
100
|
}
|
|
101
101
|
/**
|
|
102
102
|
* Advisory (never flips ok): a live clone bundle whose subject env is provisioned while its
|
|
103
|
-
* state story is undeclared probably points at state the
|
|
103
|
+
* state story is undeclared probably points at state the study does not control. Emitted at
|
|
104
104
|
* most once per bundle (the subject block is bundle-level, never per stream). GITHUB_TOKEN
|
|
105
105
|
* is mechanically excluded: the harness consumes that name for clone auth, and it carries no
|
|
106
106
|
* state implication.
|
package/dist/verify/verify.js
CHANGED
|
@@ -17,6 +17,7 @@ import { costAndReceiptFindings } from "./costs.js";
|
|
|
17
17
|
import { rerunLineageFindings } from "./rerun.js";
|
|
18
18
|
import { sharedWorldEvidenceFindings } from "./shared-world.js";
|
|
19
19
|
import { subjectStateFindings, undeclaredSubjectStateWarnings } from "./subject.js";
|
|
20
|
+
import { plural } from "../run/text.js";
|
|
20
21
|
export const VERIFY_SCHEMA = "humanish.verify-result.v1";
|
|
21
22
|
// The UNSCANNED_ARTIFACT reason names this many paths and counts the rest.
|
|
22
23
|
const MAX_LISTED_UNSCANNED = 10;
|
|
@@ -362,7 +363,7 @@ function buildShareSafety(args) {
|
|
|
362
363
|
const more = paths.length - MAX_LISTED_UNSCANNED;
|
|
363
364
|
reasons.push({
|
|
364
365
|
code: "UNSCANNED_ARTIFACT",
|
|
365
|
-
message: `The public-safety scan cannot read ${paths.length
|
|
366
|
+
message: `The public-safety scan cannot read ${plural(paths.length, "file")} as text that ${paths.length === 1 ? "is not a stream screenshot under screenshots/ or a registered recording" : "are not stream screenshots under screenshots/ or registered recordings"}: ${shown}${more > 0 ? ` and ${more} more` : ""}. Review them before sharing.`,
|
|
366
367
|
});
|
|
367
368
|
}
|
|
368
369
|
const rawStreamIds = rawScreenshotStreamIds(args.bundle, args.redactedShapeFrames);
|
|
@@ -9,7 +9,7 @@ extension seams, but public out-of-tree actor registration and its conformance c
|
|
|
9
9
|
shipped. Also not shipped: the full `Actor.run(input)` interface, `ApprovalPolicy`,
|
|
10
10
|
`StagehandCuaActor`, and the `persona-fidelity` verify check. `RedactionHooks` ships in
|
|
11
11
|
`src/evidence/redaction.ts`, and the computer-use loop takes it; the other adapters do not take it
|
|
12
|
-
yet. Of the capabilities, routing checks `lanes` and `producesScreenshots` (`src/
|
|
12
|
+
yet. Of the capabilities, routing checks `lanes` and `producesScreenshots` (`src/study/routing.ts`):
|
|
13
13
|
an actor that declares no screenshots cannot run on the computer-use or scripted-browser route.
|
|
14
14
|
Decision 6's capture-time screenshot stance was recanted in 0.6.0; see the inline notes and the
|
|
15
15
|
capture-vs-publish rule in
|
|
@@ -17,7 +17,7 @@ capture-vs-publish rule in
|
|
|
17
17
|
|
|
18
18
|
`codex-exec` is a real dispatch key for terminal-product studies, but the exported
|
|
19
19
|
descriptor `runSession` is a fail-closed compatibility entry. Live execution is
|
|
20
|
-
owned by `
|
|
20
|
+
owned by `runTerminalProductStudy`, which coordinates sandbox creation,
|
|
21
21
|
command-scoped runtime auth, evidence, caps, and by-id cleanup.
|
|
22
22
|
|
|
23
23
|
The `pi-agent-core` and `claude-agent-sdk` descriptors, the `app` run kind and the
|
|
@@ -409,7 +409,7 @@ Plan:
|
|
|
409
409
|
|
|
410
410
|
1. **Parse the whole persona** into a `ResolvedPersona`: `{ id, name, summary, goals[],
|
|
411
411
|
traits: { patience, skill, accessibilityNeeds? }, constraints[], sourceDigest }` (map
|
|
412
|
-
`technical_confidence` to `skill`). The shipped shape in `src/
|
|
412
|
+
`technical_confidence` to `skill`). The shipped shape in `src/study/persona.ts` has no
|
|
413
413
|
`goals`, adds `background`, and makes every trait optional.
|
|
414
414
|
2. **Compile traits into actor-neutral directives**, not prose, via a pure
|
|
415
415
|
`personaToDirectives(p)`:
|
|
@@ -435,7 +435,7 @@ traits: { patience, skill, accessibilityNeeds? }, constraints[], sourceDigest }`
|
|
|
435
435
|
candidate. The harness imposes no turn cap; its hard stops are the wall-clock
|
|
436
436
|
`timeoutMs` and declared spend caps.
|
|
437
437
|
4. **Bind the same directives per harness.** `renderPersonaPromptSection`
|
|
438
|
-
(`src/
|
|
438
|
+
(`src/study/persona.ts`) renders the directives as one prompt section, and each harness puts
|
|
439
439
|
that section where its model reads standing instructions:
|
|
440
440
|
- Computer use: `composeParticipantInstructions`
|
|
441
441
|
(`src/routes/computer-use/participant-prompt.ts`) opens the participant instructions
|
|
@@ -465,7 +465,7 @@ traits: { patience, skill, accessibilityNeeds? }, constraints[], sourceDigest }`
|
|
|
465
465
|
`persona-fidelity` verify check asserts that the friction and accessibility directives
|
|
466
466
|
reached the actor input and that a `gave_up` run cites a concrete friction reason (not a
|
|
467
467
|
turn count). "Did the persona drive the run" becomes a verifiable artifact. (Status
|
|
468
|
-
2026-06-11: `personaToDirectives` shipped in `src/
|
|
468
|
+
2026-06-11: `personaToDirectives` shipped in `src/study/persona.ts` and `traitsApplied` is
|
|
469
469
|
threaded on the codex-exec terminal route, but the `persona-fidelity` verify check is
|
|
470
470
|
not-yet-shipped roadmap. The computer-use route records `persona.traitsApplied` from the
|
|
471
471
|
resolved persona in `src/routes/computer-use/participant-prompt.ts`.)
|
|
@@ -61,7 +61,7 @@ admission. There is no generic CDP, command, file, navigation-management, or
|
|
|
61
61
|
runtime-management method. Initial target navigation remains an adapter-owned
|
|
62
62
|
operation.
|
|
63
63
|
|
|
64
|
-
Replies acknowledge completion or contain a finite `
|
|
64
|
+
Replies acknowledge completion or contain a finite `ComputerUseExecutorError` code and
|
|
65
65
|
`not_dispatched` / `outcome_uncertain` disposition. They never include raw
|
|
66
66
|
exception prose, typed text, page URLs, or browser errors in their error fields.
|
|
67
67
|
A generic driver exception after invocation is uncertain; only a genuine typed
|
|
@@ -31,13 +31,13 @@ dropped: `plane.exposure` must be absent (claiming synthetic on a real site is a
|
|
|
31
31
|
external-public disclosure limits are required.
|
|
32
32
|
|
|
33
33
|
Why the getHost synthetic gate is deliberately not reachable from the app-url branch: that gate
|
|
34
|
-
(`concurrentSharedWorldValidationReason` in `src/
|
|
34
|
+
(`concurrentSharedWorldValidationReason` in `src/study/validation.ts` requires `subject.exposure:
|
|
35
35
|
synthetic`, a `0.0.0.0` bind and no `keep`; verify's `getHostPlaneFindings` in
|
|
36
36
|
`src/verify/shared-world-concurrent.ts` then requires `plane.exposure == synthetic` and
|
|
37
37
|
`subject.state.provenance == seeded`) exists because a getHost URL is internet-reachable and
|
|
38
38
|
harness-owned; real data behind a harness-exposed URL is the hazard. A public site the harness
|
|
39
39
|
neither provisioned nor exposed has neither property, so the gate's hazard does not exist there. The
|
|
40
|
-
app-url branch is validated by `externalPublicSharedWorldValidationReason` (`src/
|
|
40
|
+
app-url branch is validated by `externalPublicSharedWorldValidationReason` (`src/study/validation.ts`)
|
|
41
41
|
and is reached before the getHost gate; a snapshot regression test pins the getHost path
|
|
42
42
|
byte-unchanged.
|
|
43
43
|
|
|
@@ -51,17 +51,18 @@ page, and sends `Runtime.evaluate({ url: location.href, title, text })` over the
|
|
|
51
51
|
`CuaObservation.url` is runtime-only by contract (it drives `stopWhen`/progress but is never
|
|
52
52
|
persisted raw into the trace).
|
|
53
53
|
|
|
54
|
-
The route reads each participant's URL through one callback, `
|
|
54
|
+
The route reads each participant's URL through one callback, `LoopTaps.onObservedUrl?(url)`,
|
|
55
55
|
invoked right after every `executor.observe()` (the initial observe and each loop observe) with
|
|
56
56
|
`observation.url`, threaded through `CuaActorSessionOptions` → `CuaParticipantDeps` → the concurrent
|
|
57
57
|
orchestrator. It adds no CDP code and needs no change to the lobby-trivia app. The CDP URL read is
|
|
58
58
|
unreliable on E2B desktops, so the host's code can also come from its narration or its screen
|
|
59
59
|
(step 2).
|
|
60
60
|
|
|
61
|
-
`onObservedUrl`, `onMessage` and `onScreenshot` are
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
61
|
+
`onObservedUrl`, `onMessage` and `onScreenshot` are internal `LoopTaps`
|
|
62
|
+
(`src/actors/computer-use/loop/types.ts`): 0.109.0 removed them from `ComputerUseLoopOptions`, and the
|
|
63
|
+
public `runComputerUseLoop` refuses them. The actor passes them through
|
|
64
|
+
`runComputerUseLoopWithTaps`. A library caller wraps the executor's `observe` (for `url` and
|
|
65
|
+
`screenshot`) or the provider's `nextTurn` (for `reasoning` and `message`).
|
|
65
66
|
|
|
66
67
|
Flow, a host-first barrier inside `runConcurrentSharedWorld`'s fan-out:
|
|
67
68
|
|
|
@@ -79,7 +79,7 @@ readiness, uses the labels and the `humanish_swarm` block below.
|
|
|
79
79
|
draft (`src/feedback/draft.ts`).
|
|
80
80
|
- `adapter-hardening`, `target-app-setup`, `actor-auth` and
|
|
81
81
|
`setup-quality-review`: feedback candidates that an adapter adds to the
|
|
82
|
-
bundle. `src/
|
|
82
|
+
bundle. `src/study/adapter-extension.ts` checks the value.
|
|
83
83
|
|
|
84
84
|
## Issue Body Contract
|
|
85
85
|
|
|
@@ -24,7 +24,7 @@ this preflight does not certify arbitrary hook behavior.
|
|
|
24
24
|
|
|
25
25
|
The parser reports `HUMANISH_STUDY_INVALID` with the unsupported field path. `runStudy`
|
|
26
26
|
reports `HUMANISH_STUDY_TASKS_UNSUPPORTED` in the route's failure envelope, from
|
|
27
|
-
`
|
|
27
|
+
`planStudy` (`src/study/plan.ts`) and each route's plan.
|
|
28
28
|
Refusal precedes run storage, source preparation, user hooks, local processes,
|
|
29
29
|
sandbox allocation, and model calls. No task content appears in the error.
|
|
30
30
|
|
|
@@ -24,23 +24,23 @@ fail-closed cross-validation, and forward-declared warnings.
|
|
|
24
24
|
|
|
25
25
|
## The composition
|
|
26
26
|
|
|
27
|
-
| Axis | Value
|
|
28
|
-
| --------------------------------- |
|
|
29
|
-
| `subject.source` | `terminal-product`
|
|
30
|
-
| `subject.product` | `{ name, publicSurfaces[], install?, workdir?, upload? }`: the only world the agent sees
|
|
31
|
-
| `execution.target` | `e2b-terminal` (or absent → implied)
|
|
32
|
-
| `execution.terminal` | `{ transport: exec-stream, stdin: disabled }`
|
|
33
|
-
| `execution.runtimeAuth` | `openai-env` (default) or opt-in `openai-egress`; names-only durable evidence
|
|
34
|
-
| `execution.runtime.version` | Optional exact `@openai/codex` version; observed before keyed execution
|
|
35
|
-
| `actor.model` / `reasoningEffort` | Passed to Codex as `--model` (default `gpt-5.6-sol`) and `-c model_reasoning_effort`; declarations, not observed provider identity
|
|
36
|
-
| `caps` | `{ maxUsd, maxJobs, maxMinutes }`: the blast-radius budget; `maxUsd > 0` is refused unless a `costProbe` measures spend
|
|
37
|
-
| `policies` | `allowPrivateRepoAccess` / `allowProviderCredentials` / `allowPaymentCredentials` / `allowGitHubMutation`, all default false
|
|
38
|
-
| `actor.type` | `codex-exec`: a registered terminal actor (`keyPlacement: in-sandbox-command-scoped`)
|
|
39
|
-
| route | `terminal` → `
|
|
40
|
-
|
|
41
|
-
Routing is `routeOf` (`src/
|
|
27
|
+
| Axis | Value |
|
|
28
|
+
| --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
29
|
+
| `subject.source` | `terminal-product` |
|
|
30
|
+
| `subject.product` | `{ name, publicSurfaces[], install?, workdir?, upload? }`: the only world the agent sees |
|
|
31
|
+
| `execution.target` | `e2b-terminal` (or absent → implied) |
|
|
32
|
+
| `execution.terminal` | `{ transport: exec-stream, stdin: disabled }` |
|
|
33
|
+
| `execution.runtimeAuth` | `openai-env` (default) or opt-in `openai-egress`; names-only durable evidence |
|
|
34
|
+
| `execution.runtime.version` | Optional exact `@openai/codex` version; observed before keyed execution |
|
|
35
|
+
| `actor.model` / `reasoningEffort` | Passed to Codex as `--model` (default `gpt-5.6-sol`) and `-c model_reasoning_effort`; declarations, not observed provider identity |
|
|
36
|
+
| `caps` | `{ maxUsd, maxJobs, maxMinutes }`: the blast-radius budget; `maxUsd > 0` is refused unless a `costProbe` measures spend |
|
|
37
|
+
| `policies` | `allowPrivateRepoAccess` / `allowProviderCredentials` / `allowPaymentCredentials` / `allowGitHubMutation`, all default false |
|
|
38
|
+
| `actor.type` | `codex-exec`: a registered terminal actor (`keyPlacement: in-sandbox-command-scoped`) |
|
|
39
|
+
| route | `terminal` → `runTerminalProductStudy` ([`src/routes/terminal/route.ts`](https://github.com/danielgwilson/humanish/blob/main/src/routes/terminal/route.ts)) |
|
|
40
|
+
|
|
41
|
+
Routing is `routeOf` (`src/study/plan.ts`). It sends every `terminal-product` subject to
|
|
42
42
|
this route, even with an unregistered actor, so this route refuses the actor.
|
|
43
|
-
`isTerminalProductComposition(config)` (`src/
|
|
43
|
+
`isTerminalProductComposition(config)` (`src/study/routing.ts`) also feeds the forward-declared
|
|
44
44
|
warnings.
|
|
45
45
|
|
|
46
46
|
## Repeating a terminal study with the same runtime
|
|
@@ -214,8 +214,8 @@ the cleanup proof, the interventions ledger and a minimal fail-closed cap.
|
|
|
214
214
|
## Test seams
|
|
215
215
|
|
|
216
216
|
The route reads `env` and `scorer` from its options (`src/routes/terminal/types.ts`), and a test
|
|
217
|
-
passes its seams (`desktopModule`, `renderObserver`, `now`, `costProbe`) as `
|
|
218
|
-
(`src/
|
|
217
|
+
passes its seams (`desktopModule`, `renderObserver`, `now`, `costProbe`) as `StudyDeps`
|
|
218
|
+
(`src/study/study-deps.ts`).
|
|
219
219
|
|
|
220
220
|
## The product-adapter extension seam
|
|
221
221
|
|
|
@@ -39,7 +39,7 @@ A fixture can be promoted only when:
|
|
|
39
39
|
- credential policy records env var names only, never values;
|
|
40
40
|
- adapter-owned nouns do not appear in core schema docs or core runtime code;
|
|
41
41
|
- public feedback material states that GitHub mutation was not performed;
|
|
42
|
-
- the fixture can be checked with `pnpm test tests/
|
|
42
|
+
- the fixture can be checked with `pnpm test tests/study/adapter-fixtures.test.ts`
|
|
43
43
|
and `pnpm public-surface:scan`.
|
|
44
44
|
|
|
45
45
|
## Dry-Run Web-App Fixture Shape
|
|
@@ -74,6 +74,6 @@ This keeps terminal and feedback parity independent from browser screenshots.
|
|
|
74
74
|
## Proof Commands
|
|
75
75
|
|
|
76
76
|
```bash
|
|
77
|
-
pnpm test tests/
|
|
77
|
+
pnpm test tests/study/adapter-fixtures.test.ts
|
|
78
78
|
pnpm public-surface:scan
|
|
79
79
|
```
|
package/docs/contracts/errors.md
CHANGED
|
@@ -15,7 +15,7 @@ release.
|
|
|
15
15
|
- Human mode: two or three lines on stderr.
|
|
16
16
|
|
|
17
17
|
```text
|
|
18
|
-
humanish run failed: Study not found: nope. Look in humanish/studies
|
|
18
|
+
humanish run failed: Study not found: nope. Look in humanish/studies/, or pass a .yaml path.
|
|
19
19
|
code: HUMANISH_STUDY_NOT_FOUND
|
|
20
20
|
next: humanish study list
|
|
21
21
|
```
|
|
@@ -33,13 +33,13 @@ release.
|
|
|
33
33
|
|
|
34
34
|
## Families
|
|
35
35
|
|
|
36
|
-
| Family | Codes | Emitted by
|
|
37
|
-
| ---------------------- | ----- |
|
|
38
|
-
| `HUMANISH_*` |
|
|
39
|
-
| `ANALYSIS_*` | 86 | `humanish analyze` and the analysis service (`src/analysis/`)
|
|
40
|
-
| `AUTOMATIC_ANALYSIS_*` | 21 | the post-run analysis job
|
|
41
|
-
| `analysis_*` lowercase | 29 | analysis execution
|
|
42
|
-
| `codex_*` lowercase | 13 | the Codex CLI readiness check (`src/actors/codex/`, `src/
|
|
36
|
+
| Family | Codes | Emitted by | Appears in |
|
|
37
|
+
| ---------------------- | ----- | -------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
|
|
38
|
+
| `HUMANISH_*` | 138 | every CLI command and route result | JSON `error.code`, library `error.code`, human `code:` line |
|
|
39
|
+
| `ANALYSIS_*` | 86 | `humanish analyze` and the analysis service (`src/analysis/`) | JSON `error.code` of analysis results |
|
|
40
|
+
| `AUTOMATIC_ANALYSIS_*` | 21 | the post-run analysis job | `automaticAnalysis.reason` on a run result and the job record |
|
|
41
|
+
| `analysis_*` lowercase | 29 | analysis execution | `error` fields in analysis artifacts and execution receipts |
|
|
42
|
+
| `codex_*` lowercase | 13 | the Codex CLI readiness check (`src/actors/codex/`, `src/study/doctor.ts`) | doctor rows and analysis readiness, such as `codex_unsupported_version` |
|
|
43
43
|
|
|
44
44
|
Counts are the distinct quoted codes in `src/` on 2026-10-03.
|
|
45
45
|
|
|
@@ -26,10 +26,7 @@ study: # optional, additive: which study file produced this run
|
|
|
26
26
|
id: "<study id>"
|
|
27
27
|
path: "humanish/studies/<study id>.yaml"
|
|
28
28
|
origin: "committed|ignored|explicit"
|
|
29
|
-
lab
|
|
30
|
-
id: "<study id>"
|
|
31
|
-
path: "humanish/studies/<study id>.yaml"
|
|
32
|
-
origin: "committed|ignored|explicit"
|
|
29
|
+
# Runs saved by 0.108 and earlier also have `lab:`, with the same value as `study:`.
|
|
33
30
|
source:
|
|
34
31
|
packageName: "<public package name or null>"
|
|
35
32
|
humanishSource: "present|missing"
|
|
@@ -636,6 +633,11 @@ The receipt describes the transformation; it does not attest that a participant'
|
|
|
636
633
|
finding is true. Its own bytes are not included in its hash inventory. Feedback
|
|
637
634
|
commands can subsequently generate new derivative-local artifacts.
|
|
638
635
|
|
|
636
|
+
The derivative names no sandbox. In every JSON and NDJSON file it copies, the
|
|
637
|
+
values at `sandboxId`, `subjectSandboxId` and each `providerResources[].id` read
|
|
638
|
+
`[redacted-sandbox-id]`; `sandbox-receipts.ndjson` is omitted. The source keeps
|
|
639
|
+
its raw ids, which cleanup and reclaim read.
|
|
640
|
+
|
|
639
641
|
PNG files that actor traces reference as stream screenshots are re-encoded as
|
|
640
642
|
blurred thumbnails. A PNG that nothing in `run.json` cites is omitted and
|
|
641
643
|
inventoried. Export refuses a run whose feedback candidates, adapter artifacts or
|
|
@@ -58,9 +58,10 @@ workflow without leaking private upstream truth into core.
|
|
|
58
58
|
|
|
59
59
|
## Study File
|
|
60
60
|
|
|
61
|
-
Schema: `humanish.study.v3`. `src/
|
|
62
|
-
normalized config in `src/
|
|
63
|
-
file
|
|
61
|
+
Schema: `humanish.study.v3`. `src/study/parse/study-v3.ts` rewrites a v3 document into the
|
|
62
|
+
normalized config in `src/study/types.ts`, which `src/study/config.ts` parses. humanish refuses a
|
|
63
|
+
`humanish.lab.v2` file with `HUMANISH_STUDY_V2_UNSUPPORTED`, whose message names
|
|
64
|
+
`humanish migrate <path>`; migrate converts the file to v3.
|
|
64
65
|
|
|
65
66
|
A study declares its route and composes code primitives; it is not a hardcoded kind. The
|
|
66
67
|
top-level keys:
|
|
@@ -95,7 +96,7 @@ unknown key is an error that lists the known ones; both are `HUMANISH_STUDY_INVA
|
|
|
95
96
|
`computer-use` and `shared-world` read `maxUsd` and `maxTotalUsd`; `terminal` reads `maxUsd`,
|
|
96
97
|
`maxJobs` and `maxMinutes`; `preview` and `scripted` take no `caps`.
|
|
97
98
|
|
|
98
|
-
|
|
99
|
+
`humanish migrate` maps a v2 file as follows: `actors[0]` is `actor`; `lanes` is the `participants` list; a
|
|
99
100
|
`roster` group is an entry with `count`; `count` is the count; `count` with `laneFocus` is the
|
|
100
101
|
homogeneous object; `execution.caps` (computer-use, shared-world) or `scenario.caps`
|
|
101
102
|
(terminal) is `caps`; `scenario.mode` is `mode`; `scenario.ref` is `scenario`; and
|
|
@@ -413,8 +414,8 @@ run local-only regardless of screenshot redaction. See the
|
|
|
413
414
|
[receiving contract](https://humanish.dev/docs/email-receiving).
|
|
414
415
|
|
|
415
416
|
A study's result has one schema on every route, `humanish.study-result.v1`
|
|
416
|
-
(`src/run/study-result.ts`), with `route
|
|
417
|
-
|
|
417
|
+
(`src/run/study-result.ts`), with `route` and `studyId`. 0.109.0 removed `labId`, which 0.108.0
|
|
418
|
+
wrote with the same value. Route-specific fields stay on their route's result. `humanish run`
|
|
418
419
|
with no study, `observe`, and a refusal made before a route is picked keep
|
|
419
420
|
`humanish.run-result.v1`. The evidence record stays `humanish.run-bundle.v1` in every case.
|
|
420
421
|
The computer-use result carries `plan` (the pre-flight participant table: concurrency, waves,
|
|
@@ -449,7 +450,11 @@ Study files are human-authored `.yaml` source under `humanish/studies/*.yaml` fo
|
|
|
449
450
|
committed public-safe studies, or ignored `.humanish/studies/*.yaml` /
|
|
450
451
|
`.humanish/local/studies/*.yaml` for private local dogfood. A v3 key the declared route
|
|
451
452
|
does not read is an error, so a study file never silently claims behavior that did not run.
|
|
452
|
-
A
|
|
453
|
+
A name found only in one of the three `labs/` directories is refused: a v2 file with
|
|
454
|
+
`HUMANISH_STUDY_V2_UNSUPPORTED`, and a v3 file with `HUMANISH_STUDY_RETIRED_DIRECTORY`, whose
|
|
455
|
+
message names the matching `studies/` directory to move it to. A path to a v3 file runs wherever
|
|
456
|
+
the file is. `study list --json`
|
|
457
|
+
lists the refused files in `retired`, each `{ path, code, message }`.
|
|
453
458
|
|
|
454
459
|
Committed fixture (`humanish/studies/first-run.yaml`):
|
|
455
460
|
|
|
@@ -496,14 +501,13 @@ Core-owned fields:
|
|
|
496
501
|
- `review`
|
|
497
502
|
- `feedbackCandidates`
|
|
498
503
|
- `study` (optional, additive): which study file produced the run,
|
|
499
|
-
`{ id, path?, origin? }`, where `origin` is `committed` (`humanish/studies
|
|
500
|
-
`humanish/labs/`
|
|
501
|
-
|
|
502
|
-
`
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
else `inferLegacyStudyId`, which reads the `persona.source` or `scenario.source`
|
|
504
|
+
`{ id, path?, origin? }`, where `origin` is `committed` (`humanish/studies/`; a run saved by
|
|
505
|
+
0.108 or earlier can name `humanish/labs/`), `ignored` (a local overlay), or `explicit` (a path
|
|
506
|
+
the operator passed). Runs saved by 0.108 and earlier also have `lab` with the same value;
|
|
507
|
+
0.109 writes `study` only. `study` is absent on bundles written before this contract and on
|
|
508
|
+
library callers who hand a `StudyConfig` directly; such a run has no study. Readers go through
|
|
509
|
+
`studyProvenanceOf` (`src/run/study-provenance.ts`): a valid `study`, else a valid `lab` from
|
|
510
|
+
an older run, else `inferLegacyStudyId`, which reads the `persona.source` or `scenario.source`
|
|
507
511
|
convention `study:<id>`, or `lab:<id>` from runs written before 0.108, and nothing else.
|
|
508
512
|
- `subject` (optional, additive): structured subject provenance,
|
|
509
513
|
`{ source: clone | app-url | local-tree, repo?, commit?, archiveSha256?,
|
|
@@ -1141,10 +1145,11 @@ proof claims zero on a `null` line, or when known spend exceeds the declared cap
|
|
|
1141
1145
|
`humanish.run-status.v1` is `status.json`, written inside each run directory by
|
|
1142
1146
|
every route at run start, refreshed on a fixed cadence while the run is
|
|
1143
1147
|
alive, and finalized when it ends: `{ schema, runId, state: running |
|
|
1144
|
-
finished | interrupted, mode, study?,
|
|
1145
|
-
signal?, outcome? }`. `lab
|
|
1146
|
-
`
|
|
1147
|
-
|
|
1148
|
+
finished | interrupted, mode, study?, pid, startedAt, updatedAt, completedAt?,
|
|
1149
|
+
signal?, outcome? }`. A record written by 0.108 or earlier also has `lab`, with the same value as
|
|
1150
|
+
`study`, and readers accept it.
|
|
1151
|
+
`outcome` carries the bundle's `verdict`, `participants`, `estimatedCostUsd` and
|
|
1152
|
+
`estimatedCostComplete` when the run finishes, then the result's `ok` and `execution: { succeeded,
|
|
1148
1153
|
failures: [{ kind, message }], warnings? }` once the Observer has rendered. The
|
|
1149
1154
|
verdict is what the participants experienced; `execution` is whether the run
|
|
1150
1155
|
worked as an execution (`kind` is `harness`, `provider-cleanup`,
|
|
@@ -1164,6 +1169,11 @@ the participant (or `subject`), the release warning and the
|
|
|
1164
1169
|
`humanish reclaim --run <id>` command. A local VM writes no receipt, so its
|
|
1165
1170
|
entry names the container and the `docker rm --force --volumes <container>`
|
|
1166
1171
|
command that removes it (through `limactl shell` on a Mac).
|
|
1172
|
+
`estimatedCostUsd` is the participants-and-desktops estimate and excludes
|
|
1173
|
+
analysis. `estimatedCostComplete` is the bundle's `cost.fullyEstimated`, written
|
|
1174
|
+
with it at publish: false means some participant or desktop usage has no price
|
|
1175
|
+
and the estimate is a lower bound. A record without it, as every record before
|
|
1176
|
+
0.109, says nothing about completeness, and readers treat it as unknown.
|
|
1167
1177
|
|
|
1168
1178
|
`interrupted` with `signal` (`SIGINT`, `SIGTERM` or `SIGHUP`) is written by
|
|
1169
1179
|
the CLI's run command when that signal stops a live run outside post-run
|
|
@@ -1201,7 +1211,8 @@ them is a claim about what a participant did.
|
|
|
1201
1211
|
cheapest-source-first: the `status.json` record, else the bundle, else the run
|
|
1202
1212
|
directory alone. `{ runId, derivedFrom: status | bundle | directory, liveness,
|
|
1203
1213
|
mode?, pid?, study?, startedAt?, updatedAt?, completedAt?, verdict?,
|
|
1204
|
-
participants?, estimatedCostUsd?,
|
|
1214
|
+
participants?, estimatedCostUsd?, estimatedCostComplete?, analysisCost?,
|
|
1215
|
+
durationMs? }`. The point is cost: walking
|
|
1205
1216
|
every run tree and parsing every bundle measured 167ms on a 25-run project,
|
|
1206
1217
|
against 16ms cold and 2.8ms warm here, which is what makes a surface that
|
|
1207
1218
|
refreshes on a cadence affordable. `derivedFrom` is reported so a surprising row
|
|
@@ -1210,6 +1221,14 @@ can be traced to the file it came from. A run with receipts and no outcome is
|
|
|
1210
1221
|
in-progress bundle reached without a status record: there is no freshness to
|
|
1211
1222
|
judge, and nothing in it says the run finished.
|
|
1212
1223
|
|
|
1224
|
+
`estimatedCostUsd` is the participants-and-desktops figure from the status
|
|
1225
|
+
record or bundle and excludes analysis. `estimatedCostComplete` says whether that
|
|
1226
|
+
figure is whole, and is absent when the record does not say. `analysisCost`
|
|
1227
|
+
covers the analysis: `{ requests, estimatedUsd, complete, providers }` for every
|
|
1228
|
+
analysis attempt the run sent, read by the same reader as `humanish stats`, and
|
|
1229
|
+
absent when the run sent none. Analysis is written after the status record, so
|
|
1230
|
+
`analysisCost` is cached on a fingerprint of the run's analysis directories.
|
|
1231
|
+
|
|
1213
1232
|
`humanish.run-detail.v1` is the watching projection, for one run. Who is in it
|
|
1214
1233
|
and what they are thinking: `{ runId, participants: [{ id, label, personaId?,
|
|
1215
1234
|
traits, status?, completionReason?, thought?, turns?, actions?, thoughts?,
|
|
@@ -1472,23 +1491,22 @@ bundle, and its absence is tolerated by verify.
|
|
|
1472
1491
|
|
|
1473
1492
|
`runStudy(config, options)` checks its `RunStudyOptions` against the study's route before anything
|
|
1474
1493
|
runs. A refusal comes back in the route's own result envelope, with no run directory, receipt or
|
|
1475
|
-
sandbox.
|
|
1476
|
-
`LabConfig`, the `Lab*` result and event types, the nine `Cua*` loop types,
|
|
1477
|
-
and `BrowserLabScoringContext`)
|
|
1478
|
-
|
|
1479
|
-
|
|
1480
|
-
|
|
|
1481
|
-
|
|
|
1482
|
-
| `
|
|
1483
|
-
| `
|
|
1484
|
-
| `
|
|
1485
|
-
| `
|
|
1486
|
-
| `
|
|
1487
|
-
| `
|
|
1488
|
-
| `
|
|
1489
|
-
| `
|
|
1490
|
-
| `
|
|
1491
|
-
| `rerun.participantIds` | Participants to rerun from `rerun.sourceRunId` | computer use |
|
|
1494
|
+
sandbox. 0.109.0 removed the 0.107 names (`runLab`, `RunLabOptions`, `parseLabConfig`,
|
|
1495
|
+
`LAB_CONFIG_SCHEMA`, `LabConfig`, the `Lab*` result and event types, the nine `Cua*` loop types,
|
|
1496
|
+
`CuaAdmissionLimitError` and `BrowserLabScoringContext`), which 0.108.0 kept as deprecated aliases.
|
|
1497
|
+
|
|
1498
|
+
| Option | What it does | Routes |
|
|
1499
|
+
| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
1500
|
+
| `env` | Keys and subject env for the run. Defaults to `process.env` | all |
|
|
1501
|
+
| `scorer` | An `AdapterScorerModule` (`score`, `deriveFeedback`, and `deriveArtifacts` on browser routes) over the finished evidence. A scorer written for one context passes through `browserScorer` or `terminalScorer` | computer use, shared world, terminal |
|
|
1502
|
+
| `createProvider(ctx)` | The participant's brain. `ctx` is `ProviderContext`: `config`, `participant { id, index, count }` and `executor` | computer use: E2B, local VM or in process |
|
|
1503
|
+
| `inProcess.executor({ config, appUrl })` | Drives an `app-url` or `local-app` subject in process, one participant. Requires `createProvider` | computer use |
|
|
1504
|
+
| `prepareDesktop(desktop, target)` | Runs on an E2B sandbox before provisioning. `target` is `{ kind: "subject" }` or `{ kind: "participant", participant }` | computer use on E2B (each participant); shared world (provisioned: the subject, then each participant; external-public: each participant); scripted clone (the subject) |
|
|
1505
|
+
| `onEvent(event)` | Observes `StudyEvent`s, below | all |
|
|
1506
|
+
| `onStream(event)` | Awaited. `ready` after a participant's live stream starts; its `url` carries an auth key and must never be persisted. `ended` after the participant's sandbox is gone. A rejected `ready` becomes a run warning; an `ended` rejection is swallowed. Each event carries `participantId`, `recordId` (the id of this participant's entry in `run.json` `simulations[]`, such as `sim-001`, which `streams[].simId` also holds) and `streamId`; `ready` adds `sandboxId` and `url`. 0.109.0 removed `simId`, the older name for `recordId`. | computer use and shared world on E2B |
|
|
1507
|
+
| `analysisSignal` | Cancels post-run analysis only | routes with post-run analysis |
|
|
1508
|
+
| `onObserverReady(observer)` | Awaited before any participant desktop exists | live computer use; live shared world, where the provisioned subject sandbox is already billed |
|
|
1509
|
+
| `rerun.participantIds` | Participants to rerun from `rerun.sourceRunId` | computer use |
|
|
1492
1510
|
|
|
1493
1511
|
`StudyEvent` has four types:
|
|
1494
1512
|
|
|
@@ -1515,7 +1533,7 @@ still passes it is refused with `HUMANISH_STUDY_OPTION_UNSUPPORTED`; use `rerun.
|
|
|
1515
1533
|
|
|
1516
1534
|
The route hook bags (`cuaHooks`, `scriptedHooks`, `terminalHooks`, `sharedWorldHooks`,
|
|
1517
1535
|
`automaticAnalysis`) were removed from `RunStudyOptions`, with the hook bag types and the four route
|
|
1518
|
-
runners that took them (`
|
|
1536
|
+
runners that took them (`runCuaActorStudy`, `runScriptedBrowserStudy`, `runTerminalProductStudy`,
|
|
1519
1537
|
`runConcurrentSharedWorld`). `lab` and `scorerProvenance` are set only by the humanish CLI. A
|
|
1520
1538
|
JavaScript caller that passes any of these fields is refused before anything runs. Where each
|
|
1521
1539
|
bag field's job went:
|
|
@@ -1544,11 +1562,11 @@ of it fails.
|
|
|
1544
1562
|
| ---------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
1545
1563
|
| `runDryRun`, `RunOptions`, `RunResult` | `runStudy(config, options)` on a `this-repo` study; its result is `StudyResult<"preview">` |
|
|
1546
1564
|
| `runCuaActorSession` | `runComputerUseLoop` with `createOpenAiResponsesProvider({ singleDispatch: true })`, `redaction: defaultRedactionHooks` and `now: Date.now`, which `runCuaActorSession` defaulted (the composition above). A caller that passed `desktop` and `executorOptions` writes its own `ComputerUseExecutor`: the E2B desktop executor is not exported |
|
|
1547
|
-
| `
|
|
1565
|
+
| `CuaActorStudyResult`, `ScriptedBrowserStudyResult`, `TerminalProductStudyResult`, `ConcurrentSharedWorldStudyResult` | `StudyResult<"computer-use">`, `StudyResult<"scripted">`, `StudyResult<"terminal">`, `StudyResult<"shared-world">` |
|
|
1548
1566
|
| `SubjectPhaseEvent` | the `subject-phase` `StudyEvent` through `onEvent`. Its `type` is `"subject-phase"`; the phase that was `type` is `name`, and `target` is `{ kind: "subject" }` or `{ kind: "participant", participant }`. `message`, `at`, `ok` and `durationMs` are unchanged |
|
|
1549
1567
|
| `StudyOutcome.backend`, `LabBackend` | `outcome.route` |
|
|
1550
1568
|
| `selectLabBackend`, the `routesTo*` predicates, `actorResolvesToTerminal` | `routeOf(config)` |
|
|
1551
|
-
| `
|
|
1569
|
+
| `resolveStudyDryRun` | `RunStudyOptions.dryRun`; without it, `scenario.mode` decides |
|
|
1552
1570
|
| `cuaLaneCount` | the `plan` `StudyEvent`, which lists every participant |
|
|
1553
1571
|
| `resolveSeatUrl` | `parseStudy`, which refuses a participant entry that is not same-origin loopback; build the URL with `new URL(entry, serveUrl)` |
|
|
1554
1572
|
| `cuaLaneValidationReason`, `sharedWorldValidationReason`, `concurrentSharedWorldValidationReason`, `externalPublicSharedWorldValidationReason` | `parseStudy`, which returns the same reason |
|
|
@@ -1636,8 +1654,9 @@ The `e2b-terminal` substrate is added to `RunFeedbackCandidate.substrate` so a
|
|
|
1636
1654
|
terminal-agent candidate names the substrate it ran on; browser candidates use
|
|
1637
1655
|
the existing `e2b-desktop` substrate. The routes invoke hooks over fully-assembled,
|
|
1638
1656
|
redacted evidence (`TerminalProductScoringContext` or
|
|
1639
|
-
`BrowserScoringContext`: `bundle`, runtime-only `runDir`, run identifiers
|
|
1640
|
-
actor and route metadata; all exported public types
|
|
1657
|
+
`BrowserScoringContext`: `bundle`, runtime-only `runDir`, the run identifiers
|
|
1658
|
+
`studyId` and `runId`, actor and route metadata; all exported public types; 0.109.0
|
|
1659
|
+
renamed the context's `labId` to `studyId`), scrub+redact returned
|
|
1641
1660
|
payloads, and drop any malformed score, candidate, or artifact reference with a
|
|
1642
1661
|
warning so a bad extension never poisons a verifiable bundle. On the terminal
|
|
1643
1662
|
route the context also carries `transcript`: the full normalized session
|
|
@@ -229,7 +229,10 @@ This presentation does not rewrite the saved analysis or reviewer corrections.
|
|
|
229
229
|
The frozen `humanish.observer-data.v1` schema is unchanged. A companion
|
|
230
230
|
`humanish.study-analysis.v1` artifact records source/config/input hashes,
|
|
231
231
|
participant context, evidence manifest, coverage, provider/model/prompt version,
|
|
232
|
-
status, usage and validated findings
|
|
232
|
+
status, usage and validated findings. The Observer's projection of it,
|
|
233
|
+
`observer/study-analysis.json`, also carries `spend`: what every analysis
|
|
234
|
+
request of the run cost, as [study costs](study-costs.md#one-runs-cost)
|
|
235
|
+
describes:
|
|
233
236
|
|
|
234
237
|
```text
|
|
235
238
|
.humanish/runs/<run>/
|
|
@@ -26,8 +26,23 @@ said `lab` and `labs`. Its fields keep their meanings: `totals.estimatedSpendUsd
|
|
|
26
26
|
`days[].estimatedSpendUsd`, each study's `medianCostUsd`,
|
|
27
27
|
`costSamples` and `unpricedRuns` describe participant/desktop run estimates.
|
|
28
28
|
They do not suddenly include a separate analysis request. The bundle's
|
|
29
|
-
`cost.estimatedTotalUsd` and the
|
|
30
|
-
|
|
29
|
+
`cost.estimatedTotalUsd`, `status.json`'s `outcome.estimatedCostUsd` and the run
|
|
30
|
+
index's `estimatedCostUsd` stay participants and desktops only.
|
|
31
|
+
|
|
32
|
+
## One run's cost
|
|
33
|
+
|
|
34
|
+
A run's analysis spend has one reader, `readAnalysisAccounting` in
|
|
35
|
+
`src/run/costs.ts`. Stats calls it for `costs.analysisEstimatedUsd`, and the
|
|
36
|
+
surfaces that show one run's cost read it too: the run index's `analysisCost`,
|
|
37
|
+
and the `spend` field of the Observer's companion `observer/study-analysis.json`.
|
|
38
|
+
It counts every distinct attempt the way stats does, so a run analyzed twice
|
|
39
|
+
shows two analyses. `src/run/run-cost.ts` adds that spend to the run's
|
|
40
|
+
participants-and-desktops figure, and the Observer's cost line, the Observer
|
|
41
|
+
library row's `costLabel` and the terminal UI all format the result through it.
|
|
42
|
+
The total is a lower bound, shown as `est. ~$X plus unpriced usage`, while any
|
|
43
|
+
part has an unpriced or incomplete figure. A Codex-account analysis has no
|
|
44
|
+
dollar price and reads "dollar cost unknown". An analysis that sent no request
|
|
45
|
+
adds nothing.
|
|
31
46
|
|
|
32
47
|
The new `costs` object appears on totals, each study and each day. `costsByRun`
|
|
33
48
|
contains the same accounting per selected run with stable warning codes.
|
package/examples/README.md
CHANGED
|
@@ -5,7 +5,7 @@ one from `node_modules/humanish/examples/`. Both need Node.js 22.19 or newer, an
|
|
|
5
5
|
key. `pnpm api:proof`, part of `pnpm release:check`, runs both against the packed package.
|
|
6
6
|
|
|
7
7
|
- [participant/](participant/README.md): bring your own participant. A synthetic loopback app is
|
|
8
|
-
driven through its state contract by a `
|
|
8
|
+
driven through its state contract by a `ComputerUseExecutor` and a deterministic provider, with no
|
|
9
9
|
browser, screenshots or E2B.
|
|
10
10
|
- [scorer/](scorer/README.md): score a run with your own rubric. One scorer module is attached to a
|
|
11
11
|
dry run from a library caller and from the CLI, with no desktop or running app.
|