humanish 0.107.0 → 0.108.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ARCHITECTURE.md +22 -17
- package/CHANGELOG.md +14 -0
- package/CONTEXT.md +51 -44
- package/README.md +145 -172
- package/SECURITY.md +16 -26
- package/TELEMETRY.md +8 -6
- package/dist/actors/affordance.d.ts +4 -4
- package/dist/actors/affordance.js +12 -12
- package/dist/actors/agent-session.d.ts +2 -2
- package/dist/actors/agent-session.js +7 -7
- package/dist/actors/codex/app-server-actor-trace.d.ts +1 -1
- package/dist/actors/codex/app-server-actor-trace.js +1 -1
- package/dist/actors/codex/app-server-ui.js +1 -1
- package/dist/actors/codex/codex-admission.d.ts +91 -0
- package/dist/actors/codex/codex-admission.js +150 -0
- package/dist/actors/codex/restricted-executable.d.ts +4 -4
- package/dist/actors/codex/restricted-executable.js +8 -8
- package/dist/actors/codex/restricted-launch.d.ts +2 -1
- package/dist/actors/codex/restricted-launch.js +1 -1
- package/dist/actors/codex/restricted-participant.js +8 -6
- package/dist/actors/codex/restricted-policy.d.ts +2 -2
- package/dist/actors/codex/restricted-policy.js +2 -2
- package/dist/actors/codex/restricted-session.d.ts +2 -2
- package/dist/actors/codex/restricted-session.js +10 -5
- package/dist/actors/computer-use/actor.d.ts +16 -16
- package/dist/actors/computer-use/actor.js +2 -2
- package/dist/actors/computer-use/admission-limit.d.ts +1 -1
- package/dist/actors/computer-use/admission-limit.js +1 -1
- package/dist/actors/computer-use/loop/actions.d.ts +3 -3
- package/dist/actors/computer-use/loop/actions.js +10 -10
- package/dist/actors/computer-use/loop/backstop.d.ts +3 -3
- package/dist/actors/computer-use/loop/backstop.js +14 -12
- package/dist/actors/computer-use/loop/ending.d.ts +5 -5
- package/dist/actors/computer-use/loop/ending.js +7 -7
- package/dist/actors/computer-use/loop/observation.d.ts +2 -2
- package/dist/actors/computer-use/loop/observation.js +9 -9
- package/dist/actors/computer-use/loop/provider-call.d.ts +1 -1
- package/dist/actors/computer-use/loop/provider-call.js +1 -1
- package/dist/actors/computer-use/loop/session.js +3 -3
- package/dist/actors/computer-use/loop/spend.d.ts +1 -1
- package/dist/actors/computer-use/loop/spend.js +1 -1
- package/dist/actors/computer-use/loop/trace.js +9 -8
- package/dist/actors/computer-use/loop/types.d.ts +25 -23
- package/dist/actors/computer-use/loop/usage.js +2 -2
- package/dist/actors/computer-use/openai-context.d.ts +37 -0
- package/dist/actors/computer-use/openai-context.js +214 -0
- package/dist/actors/computer-use/openai-provider.d.ts +11 -8
- package/dist/actors/computer-use/openai-provider.js +45 -34
- package/dist/actors/computer-use/openai-requests.d.ts +36 -10
- package/dist/actors/computer-use/openai-requests.js +82 -15
- package/dist/actors/computer-use/openai-wire.d.ts +19 -7
- package/dist/actors/computer-use/openai-wire.js +36 -34
- package/dist/actors/contract.d.ts +78 -43
- package/dist/actors/contract.js +29 -23
- package/dist/actors/local-agent/claude-session.d.ts +1 -1
- package/dist/actors/local-agent/claude-session.js +10 -9
- package/dist/actors/local-agent/cli.d.ts +8 -6
- package/dist/actors/local-agent/cli.js +24 -28
- package/dist/actors/local-agent/readiness.d.ts +2 -2
- package/dist/actors/local-agent/readiness.js +4 -4
- package/dist/actors/reasoning-effort.js +6 -6
- package/dist/actors/registry.d.ts +10 -10
- package/dist/actors/registry.js +18 -17
- package/dist/actors/scripted-browser/actor.d.ts +8 -8
- package/dist/actors/scripted-browser/actor.js +16 -16
- package/dist/actors/scripted-browser/browser-command.js +1 -1
- package/dist/actors/scripted-browser/steps.js +1 -1
- package/dist/actors/scripted-browser/types.d.ts +1 -1
- package/dist/actors/stop-conditions.d.ts +1 -1
- package/dist/analysis/codex-config.d.ts +6 -2
- package/dist/analysis/codex-config.js +22 -14
- package/dist/analysis/service.js +5 -5
- package/dist/cli/commands/analyze.js +19 -16
- package/dist/cli/commands/codex.js +9 -9
- package/dist/cli/commands/comms.js +54 -33
- package/dist/cli/commands/feedback.js +20 -21
- package/dist/cli/commands/lab-format.d.ts +5 -4
- package/dist/cli/commands/lab-format.js +35 -23
- package/dist/cli/commands/lab-route-computer-use.js +6 -7
- package/dist/cli/commands/lab-route-preview.js +4 -3
- package/dist/cli/commands/lab-route-shared-world.js +3 -3
- package/dist/cli/commands/lab-run.js +10 -11
- package/dist/cli/commands/lab-scorer.d.ts +2 -2
- package/dist/cli/commands/lab-scorer.js +1 -1
- package/dist/cli/commands/migrate.d.ts +4 -0
- package/dist/cli/commands/migrate.js +65 -0
- package/dist/cli/commands/observe.js +147 -108
- package/dist/cli/commands/run-command.d.ts +10 -0
- package/dist/cli/commands/run-command.js +85 -0
- package/dist/cli/commands/runs.js +137 -138
- package/dist/cli/commands/setup.d.ts +2 -2
- package/dist/cli/commands/setup.js +26 -21
- package/dist/cli/commands/study.d.ts +3 -0
- package/dist/cli/commands/study.js +182 -0
- package/dist/cli/commands/tui.d.ts +4 -4
- package/dist/cli/commands/tui.js +21 -21
- package/dist/cli/commands/watch.js +60 -75
- package/dist/cli/deprecations.d.ts +18 -0
- package/dist/cli/deprecations.js +48 -0
- package/dist/cli/doctor.d.ts +15 -5
- package/dist/cli/doctor.js +103 -48
- package/dist/cli/first-run-path.d.ts +12 -10
- package/dist/cli/first-run-path.js +30 -28
- package/dist/cli/io.d.ts +41 -6
- package/dist/cli/io.js +97 -22
- package/dist/cli/observer-follow.d.ts +2 -2
- package/dist/cli/observer-follow.js +5 -6
- package/dist/cli/orientation.d.ts +6 -3
- package/dist/cli/orientation.js +43 -39
- package/dist/cli/product-sentence.d.ts +6 -0
- package/dist/cli/product-sentence.js +6 -0
- package/dist/cli/program.d.ts +6 -1
- package/dist/cli/program.js +109 -40
- package/dist/cli/telemetry.d.ts +15 -15
- package/dist/cli/telemetry.js +48 -40
- package/dist/cli.js +1 -1
- package/dist/comms/capture-surface.d.ts +13 -12
- package/dist/comms/capture-surface.js +23 -22
- package/dist/comms/catch-host.d.ts +7 -7
- package/dist/comms/catch-host.js +14 -14
- package/dist/comms/email-catch.d.ts +3 -3
- package/dist/comms/email-catch.js +8 -8
- package/dist/comms/external-evidence.js +2 -2
- package/dist/comms/extract.js +2 -2
- package/dist/comms/fake-inbox.d.ts +2 -2
- package/dist/comms/fake-inbox.js +2 -2
- package/dist/comms/receiving-render.d.ts +1 -1
- package/dist/comms/receiving-render.js +2 -2
- package/dist/comms/sandbox-catch-script.d.ts +6 -6
- package/dist/comms/sandbox-catch-script.js +6 -6
- package/dist/comms/sandbox-catch.d.ts +36 -37
- package/dist/comms/sandbox-catch.js +34 -35
- package/dist/comms/setup.js +42 -8
- package/dist/comms/thread-evidence.d.ts +1 -1
- package/dist/comms/thread-evidence.js +6 -6
- package/dist/comms/types.d.ts +6 -6
- package/dist/comms/types.js +3 -3
- package/dist/evidence/frame-signature.d.ts +1 -1
- package/dist/evidence/frame-signature.js +4 -4
- package/dist/evidence/redaction.d.ts +3 -3
- package/dist/evidence/redaction.js +4 -4
- package/dist/feedback/draft.js +36 -78
- package/dist/feedback/export-bundle.js +1 -1
- package/dist/feedback/export.d.ts +5 -1
- package/dist/feedback/export.js +7 -8
- package/dist/feedback/feedback.d.ts +2 -2
- package/dist/feedback/feedback.js +21 -3
- package/dist/guest/chromium-text.js +2 -2
- package/dist/guest/desktop-native.d.ts +1 -1
- package/dist/guest/runtime.js +1 -1
- package/dist/index.d.ts +12 -10
- package/dist/index.js +8 -6
- package/dist/keys/key-resolution.d.ts +5 -5
- package/dist/keys/key-resolution.js +23 -23
- package/dist/lab/adapter-extension.d.ts +7 -7
- package/dist/lab/adapter-extension.js +13 -36
- package/dist/lab/adapter-scorer-loader.d.ts +7 -7
- package/dist/lab/adapter-scorer-loader.js +32 -32
- package/dist/lab/composition-rules.js +42 -42
- package/dist/lab/config.js +65 -25
- package/dist/lab/deprecated-fields.d.ts +10 -0
- package/dist/lab/deprecated-fields.js +24 -0
- package/dist/lab/device-presets.d.ts +8 -8
- package/dist/lab/device-presets.js +11 -11
- package/dist/lab/discover.d.ts +13 -9
- package/dist/lab/discover.js +117 -57
- package/dist/lab/doctor.d.ts +10 -3
- package/dist/lab/doctor.js +47 -9
- package/dist/lab/init-templates.js +147 -147
- package/dist/lab/init.d.ts +1 -1
- package/dist/lab/init.js +41 -9
- package/dist/lab/keys.d.ts +2 -0
- package/dist/lab/keys.js +42 -8
- package/dist/lab/lab-deps.d.ts +7 -12
- package/dist/lab/parse/actors.js +9 -9
- package/dist/lab/parse/comms.js +5 -5
- package/dist/lab/parse/execution.js +13 -13
- package/dist/lab/parse/study-v3.d.ts +23 -0
- package/dist/lab/parse/study-v3.js +259 -0
- package/dist/lab/parse/subject-state.d.ts +7 -7
- package/dist/lab/parse/subject-state.js +16 -16
- package/dist/lab/parse/subject.js +28 -28
- package/dist/lab/parse/values.d.ts +1 -1
- package/dist/lab/parse/values.js +3 -3
- package/dist/lab/persona-resolve.d.ts +4 -4
- package/dist/lab/persona-resolve.js +4 -4
- package/dist/lab/plan-participants.d.ts +2 -2
- package/dist/lab/plan-types.d.ts +4 -4
- package/dist/lab/plan.d.ts +1 -7
- package/dist/lab/plan.js +10 -41
- package/dist/lab/preflight-probes.js +11 -11
- package/dist/lab/preflight-result.d.ts +2 -1
- package/dist/lab/preflight-result.js +5 -4
- package/dist/lab/preflight.d.ts +7 -5
- package/dist/lab/preflight.js +5 -5
- package/dist/lab/routing.d.ts +31 -25
- package/dist/lab/routing.js +60 -29
- package/dist/lab/run-lab-events.d.ts +0 -3
- package/dist/lab/run-lab-events.js +0 -6
- package/dist/lab/run-lab-homes.d.ts +22 -10
- package/dist/lab/run-lab-homes.js +5 -1
- package/dist/lab/run-lab-options.d.ts +1 -1
- package/dist/lab/run-lab-options.js +16 -13
- package/dist/lab/summary.d.ts +7 -6
- package/dist/lab/summary.js +10 -10
- package/dist/lab/tasks.d.ts +15 -15
- package/dist/lab/tasks.js +14 -14
- package/dist/lab/types.d.ts +101 -95
- package/dist/lab/types.js +2 -0
- package/dist/lab/validation.d.ts +13 -13
- package/dist/lab/validation.js +44 -42
- package/dist/lab/warnings.d.ts +10 -0
- package/dist/lab/warnings.js +61 -33
- package/dist/library-aliases.d.ts +56 -0
- package/dist/library-aliases.js +22 -0
- package/dist/observer/artifact.d.ts +1 -1
- package/dist/observer/artifact.js +6 -6
- package/dist/observer/data.d.ts +6 -6
- package/dist/observer/data.js +5 -4
- package/dist/observer/exposure.d.ts +1 -1
- package/dist/observer/exposure.js +11 -11
- package/dist/observer/http.js +1 -1
- package/dist/observer/library.d.ts +1 -1
- package/dist/observer/library.js +4 -2
- package/dist/observer/listen.js +5 -4
- package/dist/observer/render.d.ts +1 -1
- package/dist/observer/render.js +10 -5
- package/dist/observer/run-routes.d.ts +3 -3
- package/dist/observer/run-routes.js +1 -1
- package/dist/observer/serve.js +7 -3
- package/dist/observer/tunnel.js +1 -1
- package/dist/observer-app.html +4 -4
- package/dist/routes/computer-use/bundle-parts.d.ts +9 -9
- package/dist/routes/computer-use/bundle-parts.js +15 -15
- package/dist/routes/computer-use/bundle.js +3 -3
- package/dist/routes/computer-use/e2b-desktop/comms.d.ts +1 -1
- package/dist/routes/computer-use/e2b-desktop/comms.js +14 -14
- package/dist/routes/computer-use/e2b-desktop/desktop.js +4 -4
- package/dist/routes/computer-use/e2b-desktop/fidelity.d.ts +4 -3
- package/dist/routes/computer-use/e2b-desktop/fidelity.js +11 -10
- package/dist/routes/computer-use/e2b-desktop/prepare.js +7 -7
- package/dist/routes/computer-use/e2b-desktop/start.js +9 -8
- package/dist/routes/computer-use/e2b-desktop/teardown.js +5 -4
- package/dist/routes/computer-use/external-comms.d.ts +1 -1
- package/dist/routes/computer-use/external-comms.js +1 -1
- package/dist/routes/computer-use/fanout-bundle.js +18 -18
- package/dist/routes/computer-use/fanout-records.js +10 -11
- package/dist/routes/computer-use/live-flush.d.ts +1 -1
- package/dist/routes/computer-use/live-flush.js +6 -6
- package/dist/routes/computer-use/live-phase.d.ts +4 -2
- package/dist/routes/computer-use/live-phase.js +6 -6
- package/dist/routes/computer-use/local-vm.js +1 -1
- package/dist/routes/computer-use/participant-desktop.d.ts +4 -4
- package/dist/routes/computer-use/participant-desktop.js +4 -4
- package/dist/routes/computer-use/participant-execution.d.ts +4 -4
- package/dist/routes/computer-use/participant-execution.js +9 -9
- package/dist/routes/computer-use/participant-model.d.ts +6 -0
- package/dist/routes/computer-use/participant-model.js +31 -13
- package/dist/routes/computer-use/participant-prompt.d.ts +11 -11
- package/dist/routes/computer-use/participant-prompt.js +6 -6
- package/dist/routes/computer-use/participant-runs.d.ts +7 -7
- package/dist/routes/computer-use/participant-runs.js +13 -13
- package/dist/routes/computer-use/plan.js +22 -22
- package/dist/routes/computer-use/preflight.js +21 -21
- package/dist/routes/computer-use/rerun-selection.js +1 -1
- package/dist/routes/computer-use/result.js +11 -11
- package/dist/routes/computer-use/route.d.ts +2 -1
- package/dist/routes/computer-use/route.js +15 -14
- package/dist/routes/computer-use/self-report.d.ts +7 -7
- package/dist/routes/computer-use/self-report.js +31 -31
- package/dist/routes/computer-use/setup.d.ts +1 -1
- package/dist/routes/computer-use/setup.js +17 -18
- package/dist/routes/computer-use/single-bundle.d.ts +7 -7
- package/dist/routes/computer-use/single-bundle.js +28 -30
- package/dist/routes/computer-use/subject-projection.d.ts +1 -1
- package/dist/routes/computer-use/subject-projection.js +6 -6
- package/dist/routes/computer-use/types.d.ts +40 -42
- package/dist/routes/computer-use/types.js +10 -10
- package/dist/routes/preview.d.ts +8 -2
- package/dist/routes/preview.js +8 -4
- package/dist/routes/scripted/bundle.d.ts +2 -2
- package/dist/routes/scripted/bundle.js +13 -12
- package/dist/routes/scripted/plan.d.ts +1 -1
- package/dist/routes/scripted/plan.js +9 -9
- package/dist/routes/scripted/result.js +5 -5
- package/dist/routes/scripted/route.js +5 -6
- package/dist/routes/scripted/scenario.js +2 -2
- package/dist/routes/scripted/setup.js +7 -7
- package/dist/routes/scripted/subject.js +1 -1
- package/dist/routes/scripted/surface-records.js +3 -3
- package/dist/routes/scripted/types.d.ts +3 -5
- package/dist/routes/scripted/types.js +1 -1
- package/dist/routes/shared-world/bundle-records.js +9 -8
- package/dist/routes/shared-world/bundle.d.ts +5 -5
- package/dist/routes/shared-world/bundle.js +40 -38
- package/dist/routes/shared-world/checkpoints.d.ts +1 -1
- package/dist/routes/shared-world/checkpoints.js +1 -1
- package/dist/routes/shared-world/comms.d.ts +5 -5
- package/dist/routes/shared-world/comms.js +14 -14
- package/dist/routes/shared-world/external-public.d.ts +4 -3
- package/dist/routes/shared-world/external-public.js +16 -12
- package/dist/routes/shared-world/handoff.d.ts +7 -6
- package/dist/routes/shared-world/handoff.js +65 -57
- package/dist/routes/shared-world/lobby-code.d.ts +2 -2
- package/dist/routes/shared-world/lobby-code.js +9 -9
- package/dist/routes/shared-world/participant-specs.d.ts +7 -6
- package/dist/routes/shared-world/participant-specs.js +31 -24
- package/dist/routes/shared-world/plan.js +7 -7
- package/dist/routes/shared-world/provenance.d.ts +2 -2
- package/dist/routes/shared-world/provenance.js +2 -2
- package/dist/routes/shared-world/provisioned.js +14 -13
- package/dist/routes/shared-world/result.js +20 -21
- package/dist/routes/shared-world/route.js +23 -22
- package/dist/routes/shared-world/setup.d.ts +3 -3
- package/dist/routes/shared-world/setup.js +4 -4
- package/dist/routes/shared-world/types.d.ts +30 -28
- package/dist/routes/shared-world/types.js +4 -4
- package/dist/routes/terminal/adapter.d.ts +3 -3
- package/dist/routes/terminal/adapter.js +5 -5
- package/dist/routes/terminal/bundle.d.ts +3 -3
- package/dist/routes/terminal/bundle.js +22 -21
- package/dist/routes/terminal/credentials.d.ts +4 -4
- package/dist/routes/terminal/credentials.js +7 -7
- package/dist/routes/terminal/dry-run.js +4 -4
- package/dist/routes/terminal/encoding.js +2 -2
- package/dist/routes/terminal/ledger.d.ts +3 -3
- package/dist/routes/terminal/ledger.js +14 -14
- package/dist/routes/terminal/live-finish.js +3 -3
- package/dist/routes/terminal/live-sandbox.js +12 -13
- package/dist/routes/terminal/persona.js +1 -1
- package/dist/routes/terminal/plan.js +11 -11
- package/dist/routes/terminal/recorder.js +2 -2
- package/dist/routes/terminal/result.js +8 -8
- package/dist/routes/terminal/route.js +6 -7
- package/dist/routes/terminal/sandbox.d.ts +1 -1
- package/dist/routes/terminal/sandbox.js +1 -1
- package/dist/routes/terminal/session.js +8 -7
- package/dist/routes/terminal/token-usage.d.ts +1 -1
- package/dist/routes/terminal/token-usage.js +7 -8
- package/dist/routes/terminal/trace.d.ts +2 -2
- package/dist/routes/terminal/trace.js +2 -2
- package/dist/routes/terminal/types.d.ts +26 -28
- package/dist/routes/terminal/types.js +3 -3
- package/dist/run/bundle-shape.js +3 -3
- package/dist/run/bundle.d.ts +46 -46
- package/dist/run/bundle.js +2 -1
- package/dist/run/cost-summary.js +3 -3
- package/dist/run/costs.js +7 -1
- package/dist/run/detail.d.ts +6 -6
- package/dist/run/detail.js +4 -4
- package/dist/run/dry-run-inputs.js +1 -1
- package/dist/run/dry-run.js +18 -18
- package/dist/run/judge.d.ts +19 -13
- package/dist/run/judge.js +18 -8
- package/dist/run/paths.d.ts +1 -1
- package/dist/run/paths.js +1 -1
- package/dist/run/pricing.d.ts +11 -6
- package/dist/run/pricing.js +20 -13
- package/dist/run/projection.d.ts +14 -14
- package/dist/run/projection.js +15 -15
- package/dist/run/reclaim.js +2 -2
- package/dist/run/results.d.ts +5 -5
- package/dist/run/run-index.d.ts +8 -6
- package/dist/run/run-index.js +15 -16
- package/dist/run/run-not-found.d.ts +5 -0
- package/dist/run/run-not-found.js +15 -0
- package/dist/run/run.d.ts +5 -4
- package/dist/run/run.js +1 -1
- package/dist/run/sandbox-receipts.d.ts +2 -2
- package/dist/run/sandbox-receipts.js +3 -3
- package/dist/run/shared-world-evidence.d.ts +36 -35
- package/dist/run/shared-world-evidence.js +4 -4
- package/dist/run/shared-world-shape.d.ts +1 -1
- package/dist/run/shared-world-shape.js +6 -6
- package/dist/run/stats.d.ts +12 -6
- package/dist/run/stats.js +116 -58
- package/dist/run/status.d.ts +8 -29
- package/dist/run/status.js +5 -27
- package/dist/run/stored-runs.js +2 -1
- package/dist/run/stream-shape.js +1 -1
- package/dist/run/streams.d.ts +6 -6
- package/dist/run/study-provenance.d.ts +47 -0
- package/dist/run/study-provenance.js +63 -0
- package/dist/run/study-result.d.ts +16 -0
- package/dist/run/study-result.js +9 -0
- package/dist/run/text.d.ts +2 -0
- package/dist/run/text.js +4 -0
- package/dist/run-lab.d.ts +7 -7
- package/dist/run-lab.js +1 -3
- package/dist/study/convert.d.ts +33 -0
- package/dist/study/convert.js +432 -0
- package/dist/study/field-paths.d.ts +12 -0
- package/dist/study/field-paths.js +126 -0
- package/dist/study/files.d.ts +26 -0
- package/dist/study/files.js +43 -0
- package/dist/study/migrate.d.ts +48 -0
- package/dist/study/migrate.js +408 -0
- package/dist/study/owned-files.d.ts +38 -0
- package/dist/study/owned-files.js +182 -0
- package/dist/subject/clone.d.ts +1 -1
- package/dist/subject/desktop-cli.d.ts +1 -1
- package/dist/subject/desktop-cli.js +2 -2
- package/dist/subject/local-tree-archive.js +1 -1
- package/dist/subject/local-tree.d.ts +1 -1
- package/dist/subject/node-bootstrap.js +1 -1
- package/dist/subject/runtime.js +8 -9
- package/dist/subject/serve.d.ts +1 -1
- package/dist/subject/serve.js +3 -3
- package/dist/subject/state.d.ts +1 -1
- package/dist/subject/steps.d.ts +5 -5
- package/dist/subject/steps.js +4 -4
- package/dist/substrates/command-failure.d.ts +3 -3
- package/dist/substrates/command-failure.js +3 -3
- package/dist/substrates/detached.d.ts +6 -6
- package/dist/substrates/detached.js +7 -7
- package/dist/substrates/e2b/cdp-probe.d.ts +9 -9
- package/dist/substrates/e2b/cdp-probe.js +14 -17
- package/dist/substrates/e2b/desktop-browser.d.ts +2 -2
- package/dist/substrates/e2b/desktop-browser.js +8 -8
- package/dist/substrates/e2b/desktop-cdp.d.ts +9 -9
- package/dist/substrates/e2b/desktop-cdp.js +8 -8
- package/dist/substrates/e2b/desktop-executor.d.ts +1 -1
- package/dist/substrates/e2b/desktop-executor.js +1 -1
- package/dist/substrates/e2b/desktop-geometry.d.ts +3 -3
- package/dist/substrates/e2b/desktop-geometry.js +7 -7
- package/dist/substrates/e2b/desktop-media.d.ts +1 -1
- package/dist/substrates/e2b/desktop-media.js +1 -1
- package/dist/substrates/e2b/desktop-screenshot-cleanup.d.ts +3 -3
- package/dist/substrates/e2b/desktop-screenshot-cleanup.js +3 -3
- package/dist/substrates/e2b/sandbox.d.ts +1 -1
- package/dist/substrates/e2b/sandbox.js +1 -1
- package/dist/substrates/e2b/sdk.d.ts +11 -11
- package/dist/substrates/e2b/sdk.js +6 -6
- package/dist/tui/actions.d.ts +1 -1
- package/dist/tui/actions.js +13 -13
- package/dist/tui/contract.d.ts +16 -16
- package/dist/tui/contract.js +10 -10
- package/dist/tui/launch.d.ts +4 -4
- package/dist/tui/launch.js +18 -19
- package/dist/tui/project.js +5 -5
- package/dist/tui-app.js +82 -82
- package/dist/verify/actor.d.ts +8 -8
- package/dist/verify/actor.js +36 -36
- package/dist/verify/artifacts.d.ts +2 -2
- package/dist/verify/artifacts.js +3 -3
- package/dist/verify/context-growth.d.ts +9 -0
- package/dist/verify/context-growth.js +62 -0
- package/dist/verify/costs.d.ts +6 -6
- package/dist/verify/costs.js +8 -8
- package/dist/verify/shared-world-concurrent.d.ts +2 -2
- package/dist/verify/shared-world-concurrent.js +79 -76
- package/dist/verify/shared-world.d.ts +2 -2
- package/dist/verify/shared-world.js +16 -16
- package/dist/verify/subject.d.ts +4 -4
- package/dist/verify/subject.js +18 -18
- package/dist/verify/verify.js +92 -43
- package/docs/README.md +25 -32
- package/docs/architecture/actor-contract.md +56 -87
- package/docs/architecture/desktop-sessions.md +7 -7
- package/docs/architecture/external-public-shared-world.md +61 -59
- package/docs/architecture/github-feedback-loop.md +1 -4
- package/docs/architecture/guest-desktop.md +2 -2
- package/docs/architecture/observer.md +34 -34
- package/docs/architecture/restricted-codex-analysis.md +34 -28
- package/docs/architecture/serve.md +34 -37
- package/docs/architecture/state-driven-executor.md +55 -58
- package/docs/architecture/task-protocol-support.md +5 -5
- package/docs/architecture/terminal-product-route.md +67 -107
- package/docs/contracts/README.md +24 -0
- package/docs/contracts/adapter-admission.md +3 -3
- package/docs/contracts/adapter-fixtures.md +2 -5
- package/docs/contracts/core.md +1 -3
- package/docs/contracts/errors.md +62 -0
- package/docs/contracts/feedback.md +8 -9
- package/docs/contracts/policy.md +7 -12
- package/docs/contracts/run-bundle.md +45 -44
- package/docs/contracts/schemas.md +421 -362
- package/docs/contracts/study-analysis.md +3 -3
- package/docs/contracts/study-costs.md +11 -7
- package/docs/decisions/0001-run-bundle-is-the-source-of-truth.md +1 -1
- package/docs/decisions/0002-observer-is-one-self-contained-file.md +2 -2
- package/docs/decisions/0003-contained-paths-and-cleanup-authority.md +3 -3
- package/docs/principles/actor-fidelity.md +4 -4
- package/docs/principles/engineering.md +3 -3
- package/docs/principles/invariants-and-defaults.md +38 -38
- package/docs/principles/self-driving-harness.md +1 -5
- package/docs/principles/three-roles.md +1 -1
- package/docs/product/automatic-analysis.md +7 -7
- package/examples/participant/README.md +5 -5
- package/examples/participant/run.mjs +11 -10
- package/examples/scorer/README.md +3 -3
- package/examples/scorer/run.mjs +11 -12
- package/package.json +16 -17
- package/skills/humanish/SKILL.md +60 -58
- package/skills/humanish/agents/openai.yaml +1 -1
- package/AGENTS.md +0 -99
- package/CONTRIBUTING.md +0 -148
- package/dist/actors/codex/qualified-versions.d.ts +0 -72
- package/dist/actors/codex/qualified-versions.js +0 -132
- package/dist/cli/commands/lab.d.ts +0 -3
- package/dist/cli/commands/lab.js +0 -199
- package/dist/cli/renamed-options.d.ts +0 -11
- package/dist/cli/renamed-options.js +0 -18
- package/dist/observer/static.d.ts +0 -39
- package/dist/observer/static.js +0 -369
- package/docs/architecture/comms-inbox.md +0 -122
- package/docs/architecture/desktop-recording.md +0 -80
- package/docs/architecture/local-browser-runtime.md +0 -178
- package/docs/architecture/observer-review.md +0 -144
- package/docs/architecture/participant-media.md +0 -90
- package/docs/architecture/project-layout.md +0 -62
- package/docs/architecture/real-email-receiving.md +0 -136
- package/docs/assets/humanish-drawdb-hero.png +0 -0
- package/docs/assets/humanish-observer-hero.png +0 -0
- package/docs/goals/current.md +0 -207
- package/docs/product/lobby-trivia-3player-external-public.md +0 -77
- package/docs/product/open-source-install-experience.md +0 -240
- package/docs/ramp/README.md +0 -295
- package/docs/release/open-source-readiness.md +0 -222
- package/docs/release/public-readiness-standard.md +0 -210
- package/docs/roadmap/world-class-open-source-v0.md +0 -311
package/README.md
CHANGED
|
@@ -1,60 +1,32 @@
|
|
|
1
1
|
# humanish
|
|
2
2
|
|
|
3
3
|
Synthetic user research for apps, CLIs, and agent-facing product flows.
|
|
4
|
-
Open-source and public-safe.
|
|
5
|
-
|
|
6
|
-
humanish runs studies. Realistic synthetic participants, each with its own
|
|
7
|
-
goals, patience, and skill, actually use your product on isolated desktops while
|
|
8
|
-
you watch. A study leaves verifiable evidence: screenshots, action traces,
|
|
9
|
-
per-task completion funnels, participant outcomes with the denominator
|
|
10
|
-
attached, and estimated cost lines. A fail-closed share-safety gate decides what
|
|
11
|
-
goes into feedback drafts, export bundles and a `serve --safe` library, and the
|
|
12
|
-
end of the pipeline is a public-safe feedback draft you can turn into a real
|
|
13
|
-
issue. Committed lab source lives under `humanish/`; run evidence lands under
|
|
14
|
-
gitignored `.humanish/`.
|
|
15
4
|
|
|
16
5
|
[](https://humanish.dev/demo)
|
|
17
6
|
|
|
18
|
-
**[Watch a saved run](https://humanish.dev/demo).** Eight synthetic participants
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
blocked, and `humanish analyze` turned the 13-minute recording into seven
|
|
22
|
-
findings, each linked to the capture behind it. The page replays the real
|
|
23
|
-
Observer; nothing runs from it.
|
|
24
|
-
|
|
25
|
-
**Numbers so far, every one with its run ids.** Recall on five planted defects
|
|
26
|
-
in a small task app: 15 of 15 across three live runs
|
|
27
|
-
([benchmark](https://github.com/danielgwilson/humanish/blob/main/bench/RESULTS-2026-09-04-0.76.0.md)). Precision on apps the
|
|
28
|
-
maintainer did not write: TodoMVC 5 of 6 findings confirmed against the source,
|
|
29
|
-
drawDB 11 of 12, none invented
|
|
30
|
-
([TodoMVC](https://github.com/danielgwilson/humanish/blob/main/bench/RESULTS-TODOMVC-2026-09-01.md), [drawDB](https://github.com/danielgwilson/humanish/blob/main/bench/RESULTS-DRAWDB-2026-09-01.md)).
|
|
31
|
-
Cold install to a live study: 9 of 9 fresh directories reached the goal in 108 to 200 seconds;
|
|
32
|
-
the five on the API route cost $0.16 to $0.35 each, and the four on Codex or Claude Code are
|
|
33
|
-
unpriced ([receipt](https://github.com/danielgwilson/humanish/blob/main/docs/goals/computer-use-actor/receipts/cold-install-try-live-2026-09-01.md)).
|
|
34
|
-
Same mission, different personas: keyboard-first participants reported drawDB's
|
|
35
|
-
database modal 5 of 5 times and TodoMVC's mouse-only rename 6 of 6; mouse newcomers
|
|
36
|
-
reported them 0 of 5 and 0 of 6 ([receipt](https://github.com/danielgwilson/humanish/blob/main/docs/goals/computer-use-actor/receipts/persona-axis-phone-2026-09-03.md)).
|
|
37
|
-
Planted defects are more legible than real ones and the largest cell is six runs;
|
|
38
|
-
read these as what the machinery found, not as rates for your users.
|
|
39
|
-
|
|
40
|
-

|
|
41
|
-
|
|
42
|
-
An earlier study, kept here because the image ships in the npm package: four
|
|
43
|
-
personas on [drawDB](https://github.com/drawdb-io/drawdb), a public open-source
|
|
44
|
-
database diagram editor, driven against a commit-pinned local checkout. Every participant
|
|
45
|
-
ran a real computer-use session on a hosted desktop; the captions are each persona's
|
|
46
|
-
own final report. drawDB is the application studied; it is not a humanish adopter or endorser.
|
|
7
|
+
**[Watch a saved run](https://humanish.dev/demo).** Eight synthetic participants joined one
|
|
8
|
+
lobby of a multiplayer game on its live deployment, each on its own hosted desktop. The page
|
|
9
|
+
replays the real Observer; nothing runs from it.
|
|
47
10
|
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
## Install
|
|
11
|
+
## Try it without keys
|
|
51
12
|
|
|
52
|
-
|
|
13
|
+
In a project directory, with Node.js 22.19 or newer:
|
|
53
14
|
|
|
54
15
|
```bash
|
|
55
16
|
npm install --save-dev humanish
|
|
17
|
+
npx humanish init --yes
|
|
18
|
+
npx humanish run first-run
|
|
19
|
+
npx humanish observe --run latest --open
|
|
56
20
|
```
|
|
57
21
|
|
|
22
|
+
This writes a preview study with no model calls and no spend, so you can read the evidence format
|
|
23
|
+
before you add keys. To run participants against your own app, follow the
|
|
24
|
+
[own-app guide](https://humanish.dev/docs/your-app).
|
|
25
|
+
|
|
26
|
+
[Quickstart](https://humanish.dev/docs) · [Study your app](https://humanish.dev/docs/your-app) · [What a study costs](https://humanish.dev/docs/what-a-study-costs) · [Trust boundaries](https://humanish.dev/docs/trust-boundaries) · [CLI reference](https://humanish.dev/docs/cli) · [Limits and evidence](https://humanish.dev/failure-modes)
|
|
27
|
+
|
|
28
|
+
## Run a live study
|
|
29
|
+
|
|
58
30
|
`@e2b/desktop` is the optional peer for live hosted desktops. Install it alongside
|
|
59
31
|
humanish when choosing that route (`npm install --save-dev @e2b/desktop`) so the
|
|
60
32
|
CLI can resolve it; a one-shot `npx humanish@latest` can miss the peer. The keyless
|
|
@@ -62,44 +34,46 @@ preview and local-browser setup need only `humanish`.
|
|
|
62
34
|
|
|
63
35
|
Choose how the participant runs:
|
|
64
36
|
|
|
65
|
-
| Setup
|
|
66
|
-
|
|
|
67
|
-
| `first-run` preview
|
|
68
|
-
| [Local browser study](docs/
|
|
69
|
-
| `openai-computer-use`
|
|
70
|
-
| [`local-agent`](https://humanish.dev/docs/local-agents)
|
|
37
|
+
| Setup | Participant authentication | Desktop | Automatic findings |
|
|
38
|
+
| -------------------------------------------------------------- | ------------------------------------- | ---------------------------------------------------- | ------------------------------------------------ |
|
|
39
|
+
| `first-run` preview | None; synthetic evidence only | None | No model analysis |
|
|
40
|
+
| [Local browser study](https://humanish.dev/docs/local-browser) | Codex ChatGPT login; remote inference | Linux x64 + Docker/KVM or M3+ Mac + Lima; no E2B key | Separate Codex account analyst by default |
|
|
41
|
+
| `openai-computer-use` | `OPENAI_API_KEY` | `E2B_API_KEY` + desktop SDK | Separate OpenAI request |
|
|
42
|
+
| [`local-agent`](https://humanish.dev/docs/local-agents) | Codex or Claude Code's own login | `E2B_API_KEY` + desktop SDK | Still needs `OPENAI_API_KEY`; skipped without it |
|
|
71
43
|
|
|
72
44
|
A Codex ChatGPT login can power a `local-agent` participant. It does not
|
|
73
45
|
authenticate humanish's OpenAI API requests. Choose the actor explicitly in
|
|
74
|
-
your
|
|
46
|
+
your study; installing Codex does not change an `openai-computer-use` study.
|
|
47
|
+
|
|
48
|
+
### Study your app in a local browser
|
|
75
49
|
|
|
76
50
|
For local browsers, install only `humanish` and follow the
|
|
77
|
-
[local study setup](docs/
|
|
51
|
+
[local study setup](https://humanish.dev/docs/local-browser). The first live
|
|
78
52
|
run downloads a verified runtime image; `npx humanish runtime setup` prepares it
|
|
79
53
|
ahead of time. Supported Macs use Lima instead of Docker Desktop.
|
|
80
|
-
[Optional camera and spoken conversation](docs/
|
|
81
|
-
use a separate media runtime. [Optional desktop video/audio](docs/
|
|
82
|
-
adds continuous Observer playback; screenshots remain the default. [Local captured inboxes](docs/
|
|
54
|
+
[Optional camera and spoken conversation](https://humanish.dev/docs/participant-media)
|
|
55
|
+
use a separate media runtime. [Optional desktop video/audio](https://humanish.dev/docs/desktop-recording)
|
|
56
|
+
adds continuous Observer playback; screenshots remain the default. [Local captured inboxes](https://humanish.dev/docs/comms-inbox#local-browser-studies) support email verification without mailbox-provider credentials.
|
|
83
57
|
|
|
84
|
-
For a new local
|
|
58
|
+
For a new local study, initialize with your app URL and task:
|
|
85
59
|
|
|
86
60
|
```bash
|
|
87
61
|
npx humanish init --yes \
|
|
88
62
|
--local-browser http://127.0.0.1:3000 \
|
|
89
63
|
--local-mission "Complete the primary flow and explain anything confusing"
|
|
90
|
-
npx humanish doctor --
|
|
64
|
+
npx humanish doctor --study local-browser
|
|
91
65
|
npx humanish run local-browser
|
|
92
66
|
npx humanish verify
|
|
93
67
|
```
|
|
94
68
|
|
|
95
69
|
Start your app before running the study. `verify` checks the saved run bundle, and its
|
|
96
70
|
share-safety line says whether the evidence can be shared as-is. If you already initialized this
|
|
97
|
-
project, edit `humanish/
|
|
71
|
+
project, edit `humanish/studies/local-browser.yaml` to change its URL or mission; `init`
|
|
98
72
|
preserves existing files and warns when supplied settings cannot be applied.
|
|
99
73
|
|
|
100
74
|
`init` also adds `humanish:*` scripts to `package.json`. `npm run humanish:doctor` and
|
|
101
75
|
`npm run humanish:verify` run those commands, and `humanish:watch` runs a study and keeps the
|
|
102
|
-
Observer attached. `humanish:run` is `humanish run --dry-run`: a
|
|
76
|
+
Observer attached. `humanish:run` is `humanish run --dry-run`: a dry run with no
|
|
103
77
|
browser or model, not the live study above.
|
|
104
78
|
|
|
105
79
|
`doctor` only inspects setup. It does not install or start the runtime, open a
|
|
@@ -107,63 +81,75 @@ browser, or use Codex account quota. The local study needs a supported Codex CLI
|
|
|
107
81
|
version and ChatGPT login. Linux x64 also needs local rootful Docker, KVM and
|
|
108
82
|
TUN; M3-or-newer Apple Silicon Macs need native ARM64 Node and Lima 2.2+.
|
|
109
83
|
|
|
110
|
-
|
|
111
|
-
|
|
84
|
+
### Study drawDB through the API route
|
|
85
|
+
|
|
86
|
+
Set the desktop and model keys with hidden prompts, then send one synthetic participant into the
|
|
87
|
+
included drawDB study:
|
|
112
88
|
|
|
113
89
|
```bash
|
|
114
90
|
npx humanish keys set e2b
|
|
115
91
|
npx humanish keys set openai
|
|
116
92
|
npx humanish init --yes
|
|
117
|
-
npx humanish doctor --
|
|
118
|
-
npx humanish
|
|
93
|
+
npx humanish doctor --study try-live
|
|
94
|
+
npx humanish study check try-live
|
|
119
95
|
npx humanish run try-live
|
|
120
96
|
npx humanish observe --run latest --open
|
|
121
97
|
```
|
|
122
98
|
|
|
123
99
|
Existing `E2B_API_KEY` and `OPENAI_API_KEY` environment variables also work.
|
|
124
|
-
`doctor --
|
|
100
|
+
`doctor --study` checks the selected route's local setup without launching a
|
|
125
101
|
desktop or making a model request. It reports key presence, not remote key
|
|
126
|
-
validity, model access, or quota. `
|
|
102
|
+
validity, model access, or quota. `study check` checks manifest metadata by default.
|
|
127
103
|
`try-live` clones and studies drawDB, not your project. Set the keys before `init`:
|
|
128
104
|
`init` writes `try-live` for the participant this machine can run, and it never
|
|
129
|
-
overwrites an existing
|
|
105
|
+
overwrites an existing study file. With an OpenAI key, from the environment or the
|
|
130
106
|
`keys set` store, the participant is `openai-computer-use` and its **$2 cap covers
|
|
131
107
|
estimated participant model spend**. With no OpenAI key and a signed-in Codex or
|
|
132
108
|
Claude Code, it is `local-agent`, bounded by a ten-minute session limit with no
|
|
133
|
-
dollar cap.
|
|
134
|
-
|
|
109
|
+
dollar cap. Automatic analysis after the run is a separate request;
|
|
110
|
+
[Read the results](#read-the-results) gives its limit and how to turn it off.
|
|
135
111
|
Hosted desktop time is additional. Participant caps are checked
|
|
136
112
|
between turns and are not provider billing ceilings. Allow a few minutes for
|
|
137
113
|
the app to build and the participant to work. See [budgets and privacy](https://humanish.dev/docs/budgets-and-privacy).
|
|
138
114
|
|
|
139
|
-
|
|
115
|
+
## See what studies have found
|
|
140
116
|
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
For coding agents, install the companion skill:
|
|
152
|
-
|
|
153
|
-
```bash
|
|
154
|
-
npx skills add danielgwilson/humanish --skill humanish
|
|
155
|
-
```
|
|
117
|
+
humanish runs studies. Realistic synthetic participants, each with its own
|
|
118
|
+
goals, patience, and skill, actually use your product on isolated desktops while
|
|
119
|
+
you watch. A study leaves verifiable evidence: screenshots, action traces,
|
|
120
|
+
per-task completion funnels, participant outcomes with the denominator
|
|
121
|
+
attached, and estimated cost lines. A fail-closed share-safety gate decides what
|
|
122
|
+
goes into feedback drafts, export bundles and an `observe --all --safe` library, and the
|
|
123
|
+
end of the pipeline is a public-safe feedback draft you can turn into a real
|
|
124
|
+
issue. Committed study source lives under `humanish/`; run evidence lands under
|
|
125
|
+
gitignored `.humanish/`. [ARCHITECTURE.md](ARCHITECTURE.md) traces `humanish run <study>` through
|
|
126
|
+
the code.
|
|
156
127
|
|
|
157
|
-
|
|
128
|
+
The researcher declares the study, the participant tries the product, and the
|
|
129
|
+
stakeholder reads what happened. [Three roles](docs/principles/three-roles.md)
|
|
130
|
+
explains the design; the [email-gated signup study records](https://github.com/danielgwilson/humanish/tree/main/docs/evidence/email-signup/)
|
|
131
|
+
show a completed two-participant study and a reported keyboard-accessibility finding.
|
|
158
132
|
|
|
159
|
-
|
|
133
|
+
In the [saved run](https://humanish.dev/demo), eight participants played five rounds. Six
|
|
134
|
+
reached the final standings, two were blocked, and `humanish analyze` turned the 13-minute
|
|
135
|
+
recording into seven findings, each linked to the capture behind it.
|
|
160
136
|
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
.humanish/
|
|
164
|
-
|
|
137
|
+
**Numbers so far, every one with its run ids.** Recall on five planted defects
|
|
138
|
+
in a small task app: 15 of 15 across three live runs
|
|
139
|
+
([benchmark](https://github.com/danielgwilson/humanish/blob/main/bench/RESULTS-2026-09-04-0.76.0.md)). Precision on apps the
|
|
140
|
+
maintainer did not write: TodoMVC 5 of 6 findings confirmed against the source,
|
|
141
|
+
drawDB 11 of 12, none invented
|
|
142
|
+
([TodoMVC](https://github.com/danielgwilson/humanish/blob/main/bench/RESULTS-TODOMVC-2026-09-01.md), [drawDB](https://github.com/danielgwilson/humanish/blob/main/bench/RESULTS-DRAWDB-2026-09-01.md)).
|
|
143
|
+
Cold install to a live study: 9 of 9 fresh directories reached the goal in 108 to 200 seconds;
|
|
144
|
+
the five on the API route cost $0.16 to $0.35 each, and the four on Codex or Claude Code are
|
|
145
|
+
unpriced ([study record](https://github.com/danielgwilson/humanish/blob/main/docs/evidence/computer-use/cold-install-try-live-2026-09-01.md)).
|
|
146
|
+
Same mission, different personas: keyboard-first participants reported drawDB's
|
|
147
|
+
database modal 5 of 5 times and TodoMVC's mouse-only rename 6 of 6; mouse newcomers
|
|
148
|
+
reported them 0 of 5 and 0 of 6 ([study record](https://github.com/danielgwilson/humanish/blob/main/docs/evidence/computer-use/persona-axis-phone-2026-09-03.md)).
|
|
149
|
+
Planted defects are more legible than real ones and the largest cell is six runs;
|
|
150
|
+
read these as what the machinery found, not as rates for your users.
|
|
165
151
|
|
|
166
|
-
|
|
152
|
+
## Read the results
|
|
167
153
|
|
|
168
154
|
After a run, read its findings and verification grade:
|
|
169
155
|
|
|
@@ -202,7 +188,7 @@ review:
|
|
|
202
188
|
analysis: false
|
|
203
189
|
```
|
|
204
190
|
|
|
205
|
-
`humanish run <
|
|
191
|
+
`humanish run <study>`, `humanish watch <study>`, and live
|
|
206
192
|
starts in the TUI share this default. Dry runs and unsupported routes dispatch
|
|
207
193
|
nothing. Without `OPENAI_API_KEY`, default analysis is skipped and a successful
|
|
208
194
|
recording stays successful. A default analysis whose admission estimate is over
|
|
@@ -211,10 +197,9 @@ command that runs it. Opening the TUI or Observer never starts a request.
|
|
|
211
197
|
See [automatic analysis](docs/product/automatic-analysis.md) for configuration,
|
|
212
198
|
cancellation, and failure behavior.
|
|
213
199
|
|
|
214
|
-
##
|
|
200
|
+
## Share evidence safely
|
|
215
201
|
|
|
216
|
-
humanish is designed for public repositories and public issue queues.
|
|
217
|
-
boundary is three planks, each enforced where it actually holds:
|
|
202
|
+
humanish is designed for public repositories and public issue queues.
|
|
218
203
|
|
|
219
204
|
**1. This repo and the published package are kept public-safe by CI.** Every
|
|
220
205
|
push runs a public-surface scan (secret/key/path shapes, a sha256 binary-asset
|
|
@@ -235,11 +220,11 @@ bundle, and review the actual text and pixels before sharing.
|
|
|
235
220
|
evidence over the network when you pass `--expose`, and humanish has no auth of
|
|
236
221
|
its own behind either one:
|
|
237
222
|
|
|
238
|
-
- `humanish watch <
|
|
223
|
+
- `humanish watch <study> --expose` streams a live run, which is never
|
|
239
224
|
`share_ready`, to whoever your edge admits. It requires edge auth:
|
|
240
225
|
`--tunnel ngrok --oauth google`, or `--public-url` for an edge you run. Without
|
|
241
226
|
`--allow-email` or `--allow-domain`, the OAuth edge admits any Google account.
|
|
242
|
-
- `humanish
|
|
227
|
+
- `humanish observe --all --expose` serves your run library. With edge auth and no
|
|
243
228
|
`--safe`, everyone the edge admits sees every run, raw screenshots included.
|
|
244
229
|
With `--safe`, it serves only `share_ready` runs, and edge auth is optional.
|
|
245
230
|
|
|
@@ -258,8 +243,8 @@ does not yet detect free-form PII or PHI such as names, emails,
|
|
|
258
243
|
phone numbers, dates of birth, or medical identifiers. Keeping those out depends
|
|
259
244
|
on using synthetic data and on review, so `redaction: passed` means the
|
|
260
245
|
automated secret and path scan found no matches, not that the artifact was
|
|
261
|
-
certified free of PII or PHI. A first-class PII/PHI detector is
|
|
262
|
-
|
|
246
|
+
certified free of PII or PHI. A first-class PII/PHI detector is
|
|
247
|
+
[planned](https://github.com/danielgwilson/humanish/issues/108).
|
|
263
248
|
|
|
264
249
|
`humanish verify --json` also reports `shareSafety.status`:
|
|
265
250
|
|
|
@@ -285,14 +270,16 @@ or runs a signed-in agent. The `*.live.test.ts` suites run only when a
|
|
|
285
270
|
`HUMANISH_LIVE_*` variable and real keys are set. Live behavior is checked only
|
|
286
271
|
by runs made outside CI.
|
|
287
272
|
|
|
288
|
-
**Codex versions
|
|
289
|
-
`local-agent` with Codex) and the Codex account analyst
|
|
290
|
-
|
|
291
|
-
[`
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
|
|
295
|
-
|
|
273
|
+
**Codex versions.** Codex participants (local browser studies and
|
|
274
|
+
`local-agent` with Codex) and the Codex account analyst run any stable Codex CLI release from
|
|
275
|
+
0.154.0 on, except releases humanish refuses, listed in
|
|
276
|
+
[`codex-admission.ts`](https://github.com/danielgwilson/humanish/blob/main/src/actors/codex/codex-admission.ts).
|
|
277
|
+
A new release runs the day it ships, including one Codex updated itself to. Each launch checks
|
|
278
|
+
the release's app-server schema against the fields humanish uses and refuses one that changed
|
|
279
|
+
them. Prereleases and releases below 0.154.0 are refused, and the refusal names the release and
|
|
280
|
+
why. A hosted participant on a release humanish has not tested records a warning.
|
|
281
|
+
|
|
282
|
+
**Mobile emulation.** Mobile viewport and touch flags do not certify gesture equivalence.
|
|
296
283
|
|
|
297
284
|
**Credentials.** humanish fills each provider key that is not already set from
|
|
298
285
|
the first of these sources that has it. It prints the name and source of a key it
|
|
@@ -305,12 +292,12 @@ fills from sources 2 to 4, never the value:
|
|
|
305
292
|
4. `$XDG_CONFIG_HOME/humanish/keys.env` (by default `~/.config/humanish/keys.env`),
|
|
306
293
|
which `humanish keys set` writes as plain text with mode `0600`.
|
|
307
294
|
|
|
308
|
-
A dry run reads no provider key, so `run`, `
|
|
309
|
-
for a live
|
|
295
|
+
A dry run reads no provider key, so `run`, `run` and `watch` consult sources 2 to 4 only
|
|
296
|
+
for a live study. A live run fills every key it finds and prints the ones its plan reads: its model,
|
|
310
297
|
desktop and runtime keys, its `subject.env` names, the variable
|
|
311
298
|
`comms.email.external.authTokenEnv` names, `ANTHROPIC_API_KEY` for a Claude Code participant, and
|
|
312
299
|
`OPENAI_API_KEY` when its automatic analysis runs on OpenAI. With a declared scorer it prints
|
|
313
|
-
every key it fills, since the scorer's code may read any. `
|
|
300
|
+
every key it fills, since the scorer's code may read any. `study check`, `doctor`, `tui` and the
|
|
314
301
|
`comms` commands still consult them.
|
|
315
302
|
`humanish analyze` consults them for a live OpenAI analysis, and not for `--dry-run` or the
|
|
316
303
|
Codex analyst.
|
|
@@ -323,52 +310,62 @@ never reads the subject app's own `.env` files.
|
|
|
323
310
|
**What leaves your machine.** [Trust boundaries](https://humanish.dev/docs/trust-boundaries)
|
|
324
311
|
has the providers' retention terms.
|
|
325
312
|
|
|
326
|
-
| Party | What it receives | When
|
|
327
|
-
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
328
|
-
| OpenAI | Each step's screenshot, the persona, the mission and the actions so far; after a run, selected text and captures for analysis; for an external-public shared-world
|
|
329
|
-
| Codex or Claude Code | The same screenshots and prompts, through your signed-in agent and its provider | `local-agent` participants, local browser studies, the Codex analyst
|
|
330
|
-
| E2B | A desktop or shell running your app and everything on its screen; the repository a `clone` subject names; your working tree for a `local-tree` subject, minus gitignored files, `.env*` and other secret-shaped files; the credentials you declare for the subject app; for a terminal study, your OpenAI key unless `execution.runtimeAuth: openai-egress` | Every hosted study
|
|
331
|
-
| PostHog | Telemetry events, described [below](#telemetry)
|
|
313
|
+
| Party | What it receives | When |
|
|
314
|
+
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
|
|
315
|
+
| OpenAI | Each step's screenshot, the persona, the mission and the actions so far; after a run, selected text and captures for analysis; for an external-public shared-world study, the host participant's screen while its lobby code is read; for a terminal study, the agent's requests from the E2B sandbox | `openai-computer-use` participants; automatic or `humanish analyze` runs; external-public shared-world studies; terminal studies |
|
|
316
|
+
| Codex or Claude Code | The same screenshots and prompts, through your signed-in agent and its provider | `local-agent` participants, local browser studies, the Codex analyst |
|
|
317
|
+
| E2B | A desktop or shell running your app and everything on its screen; the repository a `clone` subject names; your working tree for a `local-tree` subject, minus gitignored files, `.env*` and other secret-shaped files; the credentials you declare for the subject app; for a terminal study, your OpenAI key unless `execution.runtimeAuth: openai-egress` | Every hosted study |
|
|
318
|
+
| PostHog | Telemetry events, described [below](#check-what-telemetry-sends) | By default |
|
|
332
319
|
|
|
333
|
-
##
|
|
320
|
+
## Find the main commands
|
|
334
321
|
|
|
335
322
|
Use `npx humanish` from your project. Full arguments and options are generated
|
|
336
323
|
from the shipped CLI in the [command reference](https://humanish.dev/docs/cli).
|
|
337
324
|
|
|
338
325
|
| Command | Purpose |
|
|
339
326
|
| -------------------------------------------------------- | ------------------------------------------------------------------------- |
|
|
340
|
-
| `humanish init --yes` | Scaffold
|
|
341
|
-
| `humanish doctor --
|
|
342
|
-
| `humanish
|
|
343
|
-
| `humanish
|
|
344
|
-
| `humanish
|
|
345
|
-
| `humanish run <
|
|
346
|
-
| `humanish watch <
|
|
327
|
+
| `humanish init --yes` | Scaffold study source and ignored runtime state. |
|
|
328
|
+
| `humanish doctor --study <study> --json` | Check the selected route's setup without exposing key values or spending. |
|
|
329
|
+
| `humanish study list --json` | List available studies. |
|
|
330
|
+
| `humanish study show <study> --json` | Read a study before running it. |
|
|
331
|
+
| `humanish study check <study> --json` | Check configuration and route warnings. |
|
|
332
|
+
| `humanish run <study>` | Run the named preview or live study. |
|
|
333
|
+
| `humanish watch <study>` | Run a study with an attached Observer. |
|
|
347
334
|
| `humanish runs --json` | List local run history. |
|
|
348
335
|
| `humanish review --run latest --json` | Read an existing run's evidence. |
|
|
349
336
|
| `humanish verify --run latest --json` | Check evidence and share-safety gates. |
|
|
350
337
|
| `humanish feedback issue --run latest --repo owner/repo` | Print an eligible feedback draft. |
|
|
351
338
|
|
|
352
|
-
##
|
|
339
|
+
## Check exit codes
|
|
353
340
|
|
|
354
341
|
| Code | Meaning |
|
|
355
342
|
| ------- | ----------------------------------------------------------------------------------------- |
|
|
356
343
|
| `0` | Success. |
|
|
357
|
-
| `1` |
|
|
344
|
+
| `1` | Usage error: unknown command, unknown option, or a missing/invalid argument. |
|
|
358
345
|
| `2` | humanish domain or validation failure. Check the JSON envelope's `error.code` for detail. |
|
|
359
346
|
| `128+N` | Terminated by signal `N`: `130` for SIGINT, `143` for SIGTERM, `129` for SIGHUP. |
|
|
360
347
|
|
|
361
|
-
##
|
|
348
|
+
## Browse studies in the terminal
|
|
362
349
|
|
|
363
|
-
`humanish tui` is for a person browsing
|
|
350
|
+
`humanish tui` is for a person browsing studies and runs. It needs an
|
|
364
351
|
interactive stdin/stdout, and refuses detected coding-agent sessions even with
|
|
365
|
-
a TTY. Agents should use `
|
|
352
|
+
a TTY. Agents should use `study list --json`, `study show <study> --json`, and
|
|
366
353
|
`runs --json`. Read [TUI behavior and JSON alternatives](https://humanish.dev/docs/review-surfaces#for-coding-agents-and-scripts).
|
|
367
354
|
|
|
368
355
|
Its Connections screen (**c**) adds an AgentMail key for
|
|
369
|
-
[real email receiving](docs/
|
|
356
|
+
[real email receiving](https://humanish.dev/docs/email-receiving), which gives each participant a
|
|
370
357
|
fresh hosted inbox.
|
|
371
358
|
|
|
359
|
+
## Give coding agents the skill
|
|
360
|
+
|
|
361
|
+
For coding agents, install the companion skill:
|
|
362
|
+
|
|
363
|
+
```bash
|
|
364
|
+
npx skills add danielgwilson/humanish --skill humanish
|
|
365
|
+
```
|
|
366
|
+
|
|
367
|
+
Source: [`skills/humanish/SKILL.md`](skills/humanish/SKILL.md).
|
|
368
|
+
|
|
372
369
|
## Find more guides
|
|
373
370
|
|
|
374
371
|
- [Lab manifests](https://humanish.dev/docs/lab-manifests): source directories, route selection,
|
|
@@ -380,16 +377,11 @@ fresh hosted inbox.
|
|
|
380
377
|
model selection, dated estimates, and study and per-participant caps.
|
|
381
378
|
- [A signed-in coding agent](https://humanish.dev/docs/local-agents): Codex or Claude Code supplies
|
|
382
379
|
the participant's model on your existing plan. E2B still needs a key and bills for desktops.
|
|
383
|
-
- [The run library](https://humanish.dev/docs/review-surfaces#serve-the-run-library): `humanish
|
|
380
|
+
- [The run library](https://humanish.dev/docs/review-surfaces#serve-the-run-library): `humanish observe --all`
|
|
384
381
|
serves your runs on loopback, with authenticated remote access,
|
|
385
382
|
[live viewing from a phone](https://humanish.dev/docs/review-surfaces#watch-a-live-run-from-your-phone)
|
|
386
383
|
and share-safe public exposure.
|
|
387
384
|
|
|
388
|
-
Mobile viewport and touch flags do not certify gesture equivalence. The
|
|
389
|
-
[2026-09-05 input-conformance correction](https://github.com/danielgwilson/humanish/blob/main/docs/goals/computer-use-actor/receipts/mobile-input-conformance-2026-09-05.md)
|
|
390
|
-
qualifies the historical results from phone-sized participants: they describe humanish's
|
|
391
|
-
measured input path, not established physical-device app behavior.
|
|
392
|
-
|
|
393
385
|
## Drive an already-running local app
|
|
394
386
|
|
|
395
387
|
Use a custom executor and non-vision provider to drive your app's state contract
|
|
@@ -404,47 +396,41 @@ Run it after installing `humanish`. It makes no model calls; it proves the
|
|
|
404
396
|
integration, not persona behavior. See [state-driven local adapters](https://humanish.dev/docs/computer-use#state-driven-local-adapters)
|
|
405
397
|
for the supported library seam.
|
|
406
398
|
|
|
407
|
-
##
|
|
399
|
+
## Use humanish as a library
|
|
408
400
|
|
|
409
401
|
`import ... from "humanish"` covers four things:
|
|
410
402
|
|
|
411
|
-
- run a
|
|
412
|
-
`
|
|
403
|
+
- run a study: `runStudy`, `RunStudyOptions`, `StudyOutcome`, `StudyResult`, `StudyEvent`, `routeOf`,
|
|
404
|
+
`parseStudy`, `STUDY_SCHEMA`;
|
|
413
405
|
- read a run: `verifyRun`, `renderObserver`, `RunBundle`, `ActorTrace`;
|
|
414
|
-
- bring a participant: `
|
|
406
|
+
- bring a participant: `ComputerUseProvider`, `ComputerUseExecutor`, `ProviderContext`,
|
|
415
407
|
`createOpenAiResponsesProvider`, `runComputerUseLoop`, `defaultRedactionHooks`;
|
|
416
408
|
- score a run: `AdapterScorerModule` and its scoring contexts.
|
|
417
409
|
|
|
418
410
|
Everything else runs through the `humanish` command. The
|
|
419
411
|
[library page](https://humanish.dev/docs/library) has an example for each, and
|
|
420
|
-
[the options contract](docs/contracts/schemas.md#library-options
|
|
412
|
+
[the options contract](docs/contracts/schemas.md#library-options) lists which routes
|
|
421
413
|
take which option.
|
|
422
414
|
|
|
423
|
-
##
|
|
424
|
-
|
|
425
|
-
The researcher declares the study, the participant tries the product, and the
|
|
426
|
-
stakeholder reads what happened. [Three roles](docs/principles/three-roles.md)
|
|
427
|
-
explains the design; the [email-gated signup receipts](https://github.com/danielgwilson/humanish/tree/main/docs/goals/email-gated-signup/receipts/)
|
|
428
|
-
show a completed two-participant study and a reported keyboard-accessibility finding.
|
|
429
|
-
|
|
430
|
-
## Telemetry
|
|
415
|
+
## Check what telemetry sends
|
|
431
416
|
|
|
432
417
|
humanish sends anonymous usage events to PostHog by default. Each event carries
|
|
433
418
|
the command, the humanish version, your OS and Node major version, whether it
|
|
434
419
|
ran in CI, the exit code, a duration bucket and a random machine id made on
|
|
435
420
|
first use. A study adds whether it was a dry run, an outcome word, the
|
|
436
421
|
participant's brain route, stop and diagnostic categories, and humanish's error
|
|
437
|
-
code when it fails. The
|
|
438
|
-
`humanish init` writes; any other
|
|
422
|
+
code when it fails. The study id is sent only when it is a starter study that
|
|
423
|
+
`humanish init` writes; any other study is sent as `custom`. Your study ids,
|
|
439
424
|
subjects, personas, prompts, paths, run ids and evidence are never sent.
|
|
440
425
|
`humanish telemetry status` prints the exact event, and `humanish telemetry
|
|
441
426
|
disable` or `DO_NOT_TRACK=1` turns it off. [TELEMETRY.md](TELEMETRY.md) lists
|
|
442
427
|
every field.
|
|
443
428
|
|
|
444
|
-
##
|
|
429
|
+
## Contribute
|
|
445
430
|
|
|
446
|
-
Contributors: read
|
|
447
|
-
|
|
431
|
+
Contributors: read
|
|
432
|
+
[CONTRIBUTING.md](https://github.com/danielgwilson/humanish/blob/main/CONTRIBUTING.md) first. It
|
|
433
|
+
gives the reading order, the commands CI runs and what a pull request needs.
|
|
448
434
|
|
|
449
435
|
```bash
|
|
450
436
|
pnpm install --frozen-lockfile
|
|
@@ -457,27 +443,14 @@ Try the CLI from source:
|
|
|
457
443
|
```bash
|
|
458
444
|
pnpm humanish watch
|
|
459
445
|
pnpm humanish verify
|
|
460
|
-
pnpm humanish
|
|
446
|
+
pnpm humanish study list
|
|
461
447
|
```
|
|
462
448
|
|
|
463
|
-
|
|
464
|
-
`pnpm humanish feedback issue --repo danielgwilson/humanish`. It prints a draft and posts nothing.
|
|
465
|
-
|
|
466
|
-
## Docs
|
|
449
|
+
## Browse the docs
|
|
467
450
|
|
|
468
451
|
- [User guides and generated CLI reference](https://humanish.dev/docs)
|
|
469
|
-
- [
|
|
470
|
-
- [
|
|
471
|
-
- [Contributor and agent ramp](docs/ramp/README.md)
|
|
452
|
+
- [Contributing: reading order, commands and pull requests](https://github.com/danielgwilson/humanish/blob/main/CONTRIBUTING.md)
|
|
453
|
+
- [Contributor and agent ramp](https://github.com/danielgwilson/humanish/blob/main/docs/ramp/README.md)
|
|
472
454
|
- [Architecture: the run path, code map and invariants](ARCHITECTURE.md)
|
|
473
|
-
- [Project layout: the `humanish/` and `.humanish/` folders](docs/
|
|
455
|
+
- [Project layout: the `humanish/` and `.humanish/` folders](https://humanish.dev/docs/project-layout)
|
|
474
456
|
- [Feedback contract](docs/contracts/feedback.md)
|
|
475
|
-
- [Release readiness and gates](docs/release/open-source-readiness.md)
|
|
476
|
-
|
|
477
|
-
Dated design documents may preserve historical mechanisms. Start with the
|
|
478
|
-
current goals and the executable CLI when checking what is supported.
|
|
479
|
-
|
|
480
|
-
## Release Status
|
|
481
|
-
|
|
482
|
-
The package is published on npm. Publishing a new version requires explicit
|
|
483
|
-
maintainer authorization; see the [release procedure](docs/release/open-source-readiness.md#publish-procedure).
|
package/SECURITY.md
CHANGED
|
@@ -1,34 +1,24 @@
|
|
|
1
|
-
# Security
|
|
1
|
+
# Security policy
|
|
2
2
|
|
|
3
|
-
##
|
|
3
|
+
## Report a vulnerability privately
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
Do not file public issues that contain sensitive data. Redact the data and
|
|
10
|
-
describe the class of problem instead.
|
|
11
|
-
|
|
12
|
-
## Reporting A Vulnerability
|
|
13
|
-
|
|
14
|
-
If you find a security issue, open a minimal public issue only when the report
|
|
15
|
-
does not disclose exploitable details or sensitive data. Otherwise, contact the
|
|
16
|
-
maintainer privately through the repository owner profile.
|
|
5
|
+
Report a vulnerability through GitHub private vulnerability reporting:
|
|
6
|
+
<https://github.com/danielgwilson/humanish/security/advisories/new>. Only you and the maintainer
|
|
7
|
+
see the report until a fix ships. If you cannot use GitHub, email the maintainer at the address in
|
|
8
|
+
`package.json`'s `author` field.
|
|
17
9
|
|
|
18
10
|
Include:
|
|
19
11
|
|
|
20
|
-
- affected version or commit;
|
|
21
|
-
- command
|
|
22
|
-
- safe
|
|
23
|
-
- redacted evidence path or synthetic fixture;
|
|
24
|
-
- whether
|
|
12
|
+
- the affected version or commit;
|
|
13
|
+
- the command you ran;
|
|
14
|
+
- safe steps to reproduce it;
|
|
15
|
+
- a redacted evidence path or a synthetic fixture;
|
|
16
|
+
- whether a generated `.humanish/` artifact may contain sensitive data.
|
|
25
17
|
|
|
26
|
-
##
|
|
18
|
+
## Keep sensitive data out of public issues
|
|
27
19
|
|
|
28
|
-
|
|
20
|
+
humanish must not contain or emit PII, PHI, secrets, keys, tokens, raw private transcripts,
|
|
21
|
+
private screenshots, private customer data, private patient data or private source snippets.
|
|
29
22
|
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
pnpm public-surface:scan
|
|
33
|
-
pnpm pack:dry-run
|
|
34
|
-
```
|
|
23
|
+
Do not file a public issue that contains sensitive data. Redact it and describe the kind of
|
|
24
|
+
problem.
|