humanish 0.127.0 → 0.128.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. package/CHANGELOG.md +24 -0
  2. package/CONTEXT.md +2 -2
  3. package/README.md +1 -1
  4. package/dist/actors/closing-report.d.ts +8 -1
  5. package/dist/actors/closing-report.js +4 -1
  6. package/dist/actors/codex/restricted-participant-policy.js +2 -5
  7. package/dist/actors/computer-use/loop/debrief.js +6 -5
  8. package/dist/actors/computer-use/loop/ending.d.ts +5 -4
  9. package/dist/actors/computer-use/loop/ending.js +5 -4
  10. package/dist/actors/computer-use/loop/output-limit.js +3 -0
  11. package/dist/actors/computer-use/loop/types.d.ts +16 -10
  12. package/dist/actors/computer-use/loop.js +8 -3
  13. package/dist/actors/contract.d.ts +10 -6
  14. package/dist/cli/commands/study-format.js +13 -2
  15. package/dist/feedback/export.js +3 -2
  16. package/dist/observer/data.d.ts +13 -6
  17. package/dist/observer/data.js +40 -10
  18. package/dist/observer-app.html +8 -8
  19. package/dist/routes/computer-use/fanout-bundle.js +25 -6
  20. package/dist/routes/computer-use/participant-model.js +1 -1
  21. package/dist/run/bundle.d.ts +7 -4
  22. package/dist/run/bundle.js +10 -4
  23. package/dist/run/detail.js +8 -2
  24. package/dist/run/locate.d.ts +2 -2
  25. package/dist/run/locate.js +2 -2
  26. package/dist/run/run.js +11 -3
  27. package/dist/run/stream-trace.d.ts +17 -7
  28. package/dist/run/stream-trace.js +135 -29
  29. package/dist/run/streams.d.ts +4 -1
  30. package/dist/study/bundle-estimate.js +17 -17
  31. package/dist/verify/costs.d.ts +7 -3
  32. package/dist/verify/costs.js +27 -12
  33. package/dist/verify/verify.js +14 -8
  34. package/docs/architecture/actor-contract.md +7 -3
  35. package/docs/architecture/github-feedback-loop.md +1 -1
  36. package/docs/architecture/observer.md +19 -10
  37. package/docs/contracts/README.md +1 -1
  38. package/docs/contracts/run-bundle.md +93 -15
  39. package/docs/contracts/schemas.md +55 -45
  40. package/docs/contracts/study-analysis.md +1 -1
  41. package/docs/decisions/0002-observer-is-one-self-contained-file.md +1 -1
  42. package/package.json +1 -1
package/docs/contracts/schemas.md CHANGED
@@ -24,38 +24,38 @@ workflow without leaking private upstream truth into core.
24
24
 
25
25
  ## Contract Index
26
26
 
27
- | Contract | Schema | Public-safe fixture |
28
- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------- |
29
- | Run bundle | `humanish.run-bundle.v3` (each participant's trace in its own file, indexed by `streams[].trace`, and a running participant's items so far in a live file indexed by `streams[].liveActor.itemsFile`; v2 bundles, with running items inline, and v1 bundles, with traces inline, are still read) | `synthetic-run-bundle` |
30
- | Adapter | `humanish.adapter.v1` | `synthetic-cli-adapter` |
31
- | Study file | `humanish.study.v3` | `first-run` |
32
- | Persona | `humanish.persona.v1` | `synthetic-maintainer` |
33
- | Scenario | `humanish.scenario.v1` | `first-run-smoke` |
34
- | Actor trace | `humanish.actor-trace.v1` | `synthetic-actor-trace` |
35
- | Substrate | reserved (never shipped) | none |
36
- | Evidence stream | reserved (streams live inside the run bundle) | see [`run-bundle.md`](run-bundle.md) |
37
- | Review | `humanish.review.v1` | `contract-proof-review` |
38
- | Study analysis | `humanish.study-analysis.v1` (prompt `study-evidence-7` adds `findings[].headline`, `findings[].experience` and `designFindings[]`; older artifacts omit them; `study-evidence-8` adds participant impressions to the instructions, with no field change) | see [study analysis](study-analysis.md) and synthetic analysis fixtures in `tests/` |
39
- | Study analysis correction | `humanish.study-analysis-correction.v1` | see [study analysis](study-analysis.md#human-review-and-sharing) |
40
- | Analysis findings view | `humanish.analysis-findings.v1` (`review --json` field `analysis`; read-only projection of the selected analysis) | see [study analysis](study-analysis.md#reading-findings) |
41
- | Analysis execution receipt | `humanish.analysis-execution.v1` | see [study analysis](study-analysis.md#durable-records) |
42
- | Analysis execution start | `humanish.analysis-execution-start.v1` | see [study analysis](study-analysis.md#durable-records) |
43
- | Verification | `humanish.verify-result.v1` | `five-check-verify` |
44
- | Policy | `humanish.policy.v1` (fixture-only; not engine-validated) | `public-safety-policy` |
45
- | Feedback | `humanish.feedback.v1` | `public-safe-feedback` |
46
- | Terminal cost ledger | `humanish.terminal-cost-ledger.v1` | see Terminal Cost Ledger below |
47
- | Terminal no-spend proof | `humanish.terminal-no-spend-proof.v1` | see Terminal Cost Ledger below |
48
- | Pricing (operator-editable rates) | `humanish.pricing.v1` (`src/run/pricing.ts`; dated per-model + E2B desktop rates) | see Run Cost Summary And Estimated Actor Cost below |
49
- | Run cost summary | `humanish.run-cost-summary.v1` (additive `RunBundle.cost`; estimate, never a charge) | see Run Cost Summary And Estimated Actor Cost below |
50
- | Estimated actor cost | `humanish.actor-estimated-cost.v1` (additive `ActorTrace.estimatedCost`) | see Run Cost Summary And Estimated Actor Cost below |
51
- | Model settings | `humanish.model-settings.v1` (additive `ActorTrace.modelSettings`; the reasoning effort the request carried) | see Actor Trace below |
52
- | Affordance use | `humanish.affordance-use.v1` (additive `ActorTrace.affordanceUse`; per-class counts of the routes an actor took) | see Affordance Use below |
53
- | Adapter score | `humanish.adapter-score.v1` (`RunBundle.adapterScore`; namespaced; route-specific acceptance semantics) | see Product-Adapter Extension Seam below |
54
- | Adapter artifact | `humanish.adapter-artifact.v1` (`RunBundle.adapterArtifacts[]`; namespaced; local relative proof references) | see Product-Adapter Extension Seam below |
55
- | Shared-world evidence | `humanish.shared-world.v1` (additive `RunBundle.sharedWorld` + `RunBundle.attributionClass`; `topologyMode: sequential \| concurrent`) | see Shared-World Evidence below |
56
- | Comms thread | `humanish.comms-thread.v1` (off-app email/SMS the app sent, captured; a `kind: log` run-dir artifact of digests only: from/to/subject/link digests + one OTP count, never raw) | see `comms` under Study File |
57
- | Serve result | `humanish.serve-result.v1` (`src/observer/serve.ts` is authoritative) | none (command result envelope; see Serve Result below) |
58
- | Serve control plane | reserved (`/_humanish/api/*` answers `501` `HUMANISH_SERVE_CONTROL_PLANE_DISABLED` in v1) | none |
27
+ | Contract | Schema | Public-safe fixture |
28
+ | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- |
29
+ | Run bundle | `humanish.run-bundle.v4` (each participant's trace in its own file, indexed by `streams[].trace`, with its per-request lists only there; a running participant's items so far in a live file indexed by `streams[].liveActor.itemsFile`, and its totals and `estimatedCost` in run.json; v3 bundles, with the lists in run.json too, v2 bundles, with running items inline, and v1 bundles, with traces inline, are still read) | `synthetic-run-bundle` |
30
+ | Adapter | `humanish.adapter.v1` | `synthetic-cli-adapter` |
31
+ | Study file | `humanish.study.v3` | `first-run` |
32
+ | Persona | `humanish.persona.v1` | `synthetic-maintainer` |
33
+ | Scenario | `humanish.scenario.v1` | `first-run-smoke` |
34
+ | Actor trace | `humanish.actor-trace.v1` | `synthetic-actor-trace` |
35
+ | Substrate | reserved (never shipped) | none |
36
+ | Evidence stream | reserved (streams live inside the run bundle) | see [`run-bundle.md`](run-bundle.md) |
37
+ | Review | `humanish.review.v1` | `contract-proof-review` |
38
+ | Study analysis | `humanish.study-analysis.v1` (prompt `study-evidence-7` adds `findings[].headline`, `findings[].experience` and `designFindings[]`; older artifacts omit them; `study-evidence-8` adds participant impressions to the instructions, with no field change) | see [study analysis](study-analysis.md) and synthetic analysis fixtures in `tests/` |
39
+ | Study analysis correction | `humanish.study-analysis-correction.v1` | see [study analysis](study-analysis.md#human-review-and-sharing) |
40
+ | Analysis findings view | `humanish.analysis-findings.v1` (`review --json` field `analysis`; read-only projection of the selected analysis) | see [study analysis](study-analysis.md#reading-findings) |
41
+ | Analysis execution receipt | `humanish.analysis-execution.v1` | see [study analysis](study-analysis.md#durable-records) |
42
+ | Analysis execution start | `humanish.analysis-execution-start.v1` | see [study analysis](study-analysis.md#durable-records) |
43
+ | Verification | `humanish.verify-result.v1` | `five-check-verify` |
44
+ | Policy | `humanish.policy.v1` (fixture-only; not engine-validated) | `public-safety-policy` |
45
+ | Feedback | `humanish.feedback.v1` | `public-safe-feedback` |
46
+ | Terminal cost ledger | `humanish.terminal-cost-ledger.v1` | see Terminal Cost Ledger below |
47
+ | Terminal no-spend proof | `humanish.terminal-no-spend-proof.v1` | see Terminal Cost Ledger below |
48
+ | Pricing (operator-editable rates) | `humanish.pricing.v1` (`src/run/pricing.ts`; dated per-model + E2B desktop rates) | see Run Cost Summary And Estimated Actor Cost below |
49
+ | Run cost summary | `humanish.run-cost-summary.v1` (additive `RunBundle.cost`; estimate, never a charge) | see Run Cost Summary And Estimated Actor Cost below |
50
+ | Estimated actor cost | `humanish.actor-estimated-cost.v1` (additive `ActorTrace.estimatedCost`) | see Run Cost Summary And Estimated Actor Cost below |
51
+ | Model settings | `humanish.model-settings.v1` (additive `ActorTrace.modelSettings`; the reasoning effort the request carried) | see Actor Trace below |
52
+ | Affordance use | `humanish.affordance-use.v1` (additive `ActorTrace.affordanceUse`; per-class counts of the routes an actor took) | see Affordance Use below |
53
+ | Adapter score | `humanish.adapter-score.v1` (`RunBundle.adapterScore`; namespaced; route-specific acceptance semantics) | see Product-Adapter Extension Seam below |
54
+ | Adapter artifact | `humanish.adapter-artifact.v1` (`RunBundle.adapterArtifacts[]`; namespaced; local relative proof references) | see Product-Adapter Extension Seam below |
55
+ | Shared-world evidence | `humanish.shared-world.v1` (additive `RunBundle.sharedWorld` + `RunBundle.attributionClass`; `topologyMode: sequential \| concurrent`) | see Shared-World Evidence below |
56
+ | Comms thread | `humanish.comms-thread.v1` (off-app email/SMS the app sent, captured; a `kind: log` run-dir artifact of digests only: from/to/subject/link digests + one OTP count, never raw) | see `comms` under Study File |
57
+ | Serve result | `humanish.serve-result.v1` (`src/observer/serve.ts` is authoritative) | none (command result envelope; see Serve Result below) |
58
+ | Serve control plane | reserved (`/_humanish/api/*` answers `501` `HUMANISH_SERVE_CONTROL_PLANE_DISABLED` in v1) | none |
59
59
 
60
60
  ## Study File
61
61
 
@@ -523,7 +523,7 @@ A study's result has one schema on every route, `humanish.study-result.v1`
523
523
  (`src/run/study-result.ts`), with `route` and `studyId`. 0.109.0 removed `labId`, which 0.108.0
524
524
  wrote with the same value. Route-specific fields stay on their route's result. `humanish run`
525
525
  with no study, `observe`, and a refusal made before a route is picked keep
526
- `humanish.run-result.v1`. The evidence record stays `humanish.run-bundle.v3` in every case.
526
+ `humanish.run-result.v1`. The evidence record stays `humanish.run-bundle.v4` in every case.
527
527
  The computer-use result carries `plan` (the pre-flight participant table: concurrency, waves,
528
528
  per-participant session budget, worst-case sandbox-minutes), `lanes[]` (always present,
529
529
  length 1 at N=1; per-participant status/session/sandbox/subject), and `laneSummary`
@@ -668,7 +668,7 @@ Adapter-owned fields:
668
668
  Abbreviated synthetic fixture (omits `cwd`, `simulations`, `streams` and `events`):
669
669
 
670
670
  ```yaml
671
- schema: humanish.run-bundle.v3
671
+ schema: humanish.run-bundle.v4
672
672
  runId: synthetic-run-bundle-2026-06-02t10-00-00-000z-proof
673
673
  mode: dry-run
674
674
  simCount: 1
@@ -1010,6 +1010,11 @@ text. Older receipts without it remain valid. The phase does not replace the
1010
1010
  separate dispatch, usage, or cleanup evidence, and does not establish the cause
1011
1011
  of a timeout. Participant outcome text includes the phase when available.
1012
1012
 
1013
+ Three fields gain one entry per model request: `tokenUsage.turns`, `conversation.requests` and
1014
+ `providerRequests`. From run bundle v4 they are only in the participant's trace file, and
1015
+ run.json's `streams[].actor` and `streams[].liveActor` leave them out; the trace loader puts them
1016
+ back ([run bundle contract](run-bundle.md#per-request-lists)).
1017
+
1013
1018
  Actors execute or simulate the trial. Actor evidence is the provider-neutral
1014
1019
  `humanish.actor-trace.v1` (`src/actors/contract.ts`): Codex app-server items,
1015
1020
  computer-use cycles, scripted browser steps, and in-sandbox terminal-agent exec
@@ -1500,8 +1505,9 @@ Three new `.v1` schema tags ship, all additive and optional so
1500
1505
  the magnitude; the flag propagates into every estimate so a stand-in is never
1501
1506
  mistaken for a live rate. An unknown model/desktop rate is declared absent
1502
1507
  (`estimatedCostUsd: null` + a `reason`), never guessed.
1503
- - `humanish.actor-estimated-cost.v1` (`ActorTrace.estimatedCost`): one participant's
1504
- token-derived model cost, with `estimatedCostUsd` (or `null` + `reason`
1508
+ - `humanish.actor-estimated-cost.v1` (`ActorTrace.estimatedCost`, and from run bundle v4
1509
+ `streams[].liveActor.estimatedCost`, which the run writer prices from a running participant's
1510
+ usage rows before leaving them out of run.json): one participant's token-derived model cost, with `estimatedCostUsd` (or `null` + `reason`
1505
1511
  `no_rate_for_model`/`no_token_usage`/`account_billing_unknown`), `ratesAsOf`, `source`, `modelId`,
1506
1512
  optional `placeholder`, and a `breakdown`. `basis: aggregated_turns_base_rate` marks usage that
1507
1513
  arrived as turn totals, such as Codex `turn.completed`, priced at the base tier because no single
@@ -1585,11 +1591,14 @@ across the study's participants. Either, both, or neither may be declared. Witho
1585
1591
  shared threshold, N participants each have their own `maxUsd`; N × `maxUsd` is the sum
1586
1592
  of those thresholds, not a guaranteed total spending ceiling.
1587
1593
 
1588
- The loop checks reported usage after a model response, before dispatching that
1589
- response's actions or requesting another turn. It does not reserve the next
1590
- request's worst-case cost. In-flight requests and concurrent participants can exceed
1591
- these estimates. These thresholds are not hard provider billing caps and
1592
- exclude desktop and target-app charges.
1594
+ The loop checks both thresholds after a model response reports usage, before
1595
+ dispatching that response's actions, and again before each provider request,
1596
+ the first included. The check before a request stops a participant whose
1597
+ sibling crossed `maxTotalUsd` while it was acting or while its desktop was
1598
+ starting. Neither check reserves the next request's worst-case cost. A request
1599
+ already in flight, at most one per running participant, can exceed these
1600
+ estimates. These thresholds are not hard provider billing caps and exclude
1601
+ desktop and target-app charges.
1593
1602
 
1594
1603
  A capped session sends a further request only while every earlier request's
1595
1604
  usage is known or bounded:
@@ -1617,8 +1626,9 @@ only; `tokenUsage` and the cost summary keep the reported usage.
1617
1626
  Crossing `maxUsd` ends `budget_reached` / `incomplete`, including when no
1618
1627
  material action has executed. The recorded reason distinguishes prior progress
1619
1628
  from no material progress; a harness budget stop is not participant abandonment.
1620
- Crossing the shared study threshold ends `budget_reached` / `incomplete`, with
1621
- sibling participants stopping when their next post-response check sees it. Reaching a
1629
+ Crossing the shared study threshold ends `budget_reached` / `incomplete`. Each
1630
+ sibling participant stops before it sends its next request; one with a request
1631
+ in flight stops when that reply arrives, before its actions run. Reaching a
1622
1632
  threshold is not proof of task completion.
1623
1633
 
1624
1634
  An absent threshold is uncapped. A zero threshold can still permit a paid model
@@ -1628,7 +1638,7 @@ mode. Use the keyless `humanish run first-run` preview or an explicit
1628
1638
  threshold on a model `src/run/pricing.ts` cannot price is refused at preflight
1629
1639
  (`HUMANISH_COMPUTER_USE_UNPRICED_CAP`, or `HUMANISH_SHARED_WORLD_UNPRICED_CAP` on a
1630
1640
  shared-world study) before sandbox allocation. This rate-availability
1631
- check is separate from the post-response spend check.
1641
+ check is separate from the spend checks during the session.
1632
1642
 
1633
1643
  Library callers can make a capped session stricter by composing it from the loop and the default
1634
1644
  provider:
@@ -1907,7 +1917,7 @@ including existence for referenced adapter artifacts.
1907
1917
 
1908
1918
  Reserved: `humanish.evidence-stream.v1` has never shipped as a standalone
1909
1919
  schema, and streams are not standalone artifacts. They are the `streams` array
1910
- inside the run bundle (`humanish.run-bundle.v3`), normalizing UI, browser, terminal, TUI,
1920
+ inside the run bundle (`humanish.run-bundle.v4`), normalizing UI, browser, terminal, TUI,
1911
1921
  code-agent UI, artifact, and summary streams, each with transport, terminal
1912
1922
  tail, completion, meaningful-use verdicts, and artifact pointers. See
1913
1923
  [`run-bundle.md`](run-bundle.md#completion-and-meaningful-use-verdicts) for
package/docs/contracts/study-analysis.md CHANGED
@@ -367,7 +367,7 @@ and an impression alone supports only a finding about what the participant said.
367
367
 
368
368
  ## Durable records
369
369
 
370
- The `humanish.observer-data.v2` schema is unchanged by analysis. A companion
370
+ The `humanish.observer-data.v3` schema is unchanged by analysis. A companion
371
371
  `humanish.study-analysis.v1` artifact records source/config/input hashes,
372
372
  participant context, evidence manifest, coverage, provider/model/prompt version,
373
373
  status, usage and validated findings. The Observer's projection of it,
package/docs/decisions/0002-observer-is-one-self-contained-file.md CHANGED
@@ -13,7 +13,7 @@ exactly those cases.
13
13
  The `observer/` workspace builds with Vite into a single HTML file with fonts inlined and no
14
14
  network references. The root build copies it to `dist/observer-app.html`. `renderObserverHtml`
15
15
  is the one path every surface (observe, observe --all, watch, studies) uses; it injects the run's
16
- `humanish.observer-data.v2` snapshot into that file, with the participants' trace items beside it
16
+ `humanish.observer-data.v3` snapshot into that file, with the participants' trace items beside it
17
17
  gzip-compressed (docs/architecture/observer.md). There is no second renderer and no feature
18
18
  flag; rolling back means pinning an older package version.
19
19
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "humanish",
3
- "version": "0.127.0",
3
+ "version": "0.128.0",
4
4
  "description": "Synthetic user research for apps, CLIs, and agent-facing product flows.",
5
5
  "keywords": [
6
6
  "cli",