humanish 0.127.1 → 0.128.0

package/CHANGELOG.md

@@ -8,6 +8,19 @@ The Unreleased section holds the full notes for the next version until it is tag
 
 ## Unreleased
 
+## 0.128.0: Run bundle v4 keeps per-request lists in trace files; the study budget is checked before each request (2026-10-13)
+
+humanish 0.128.0 keeps run.json small in long studies. Run bundles are now
+`humanish.run-bundle.v4`: each participant's per-request lists (`tokenUsage.turns`,
+`conversation.requests` and `providerRequests`) are only in its trace file, and run.json keeps the
+totals. A synthetic run of 30 participants making 2,880 requests each wrote run.json at 93 to 106
+KB, where 0.127.1 wrote 16.1, 27.2 and 41.8 MB; analysis reads run.json only up to 16 MiB. This
+release reads v1 to v4 bundles, and `loadRunBundle` joins the lists back. 0.127.1 and earlier fail
+verify on a v4 run. A computer-use participant now checks the study budget before each model
+request, so one whose sibling crossed `caps.maxTotalUsd` no longer sends one more request.
+
+[Release notes](https://github.com/danielgwilson/humanish/releases/tag/v0.128.0)
+
 ## 0.127.1: Clock-end closing reports declare their outcome; skipped participants named as skipped (2026-10-13)
 
 humanish 0.127.1 fixes two things the 0.127.0 release smoke found in scheduled studies. An OpenAI

package/CONTEXT.md

@@ -12,7 +12,7 @@ bundle and output fields keep older spellings; the second table maps them.
 | Term                | Meaning                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | Where it lives in the code                                                                                                                                                                                                                                                                                                                                                                                  |
 | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | study               | What a user designs and runs: one YAML file that declares the subject, the actor, the participants and the caps. The file lives at `humanish/studies/<id>.yaml` with schema `humanish.study.v3`, read by `humanish study` and `--study`. humanish refuses a 0.107 `humanish.lab.v2` file, and `humanish migrate` converts it (second table).                                                                                                                                                                                          | `src/study/`; `StudyConfig` in `src/study/types.ts`, parsed by `parseStudy` in `src/study/config.ts`                                                                                                                                                                                                                                                                                                        |
-| run                 | One execution of a study, on disk: `.humanish/runs/<runId>/`, with `run.json` (`humanish.run-bundle.v3`) as the source of truth and each participant's trace in its own file.                                                                                                                                                                                                                                                                                                                                                         | `src/run/`; `RunBundle` in `src/run/bundle.ts`                                                                                                                                                                                                                                                                                                                                                              |
+| run                 | One execution of a study, on disk: `.humanish/runs/<runId>/`, with `run.json` (`humanish.run-bundle.v4`) as the source of truth and each participant's trace in its own file.                                                                                                                                                                                                                                                                                                                                                         | `src/run/`; `RunBundle` in `src/run/bundle.ts`                                                                                                                                                                                                                                                                                                                                                              |
 | dry run             | A run with no live participant, from `--dry-run` or `mode: dry-run`. humanish writes a synthetic run bundle, review and Observer with no browser, model, keys or provider spend, and tests no product behavior. The `preview` route only dry-runs; "preview" names that route and nothing else.                                                                                                                                                                                                                                       | `runDryRun` in `src/run/dry-run.ts`; `resolveStudyDryRun` in `src/study/plan.ts`                                                                                                                                                                                                                                                                                                                            |
 | participant         | One simulated person, or agent, taking part in a study. The planner builds each one's record (id, persona id, assignment, labels and the route's own fields). Shared-world participants resolve the plan's records; computer-use participants rebuild the same records from the study config with `computerUseParticipants` (`src/study/plan-participants.ts`) and resolve those. `ResolvedParticipant` adds the bundle ids, persona and prompt. `run.json` records each one as a `simulations[]` entry with one or more `streams[]`. | `ResolvedParticipant` in `src/run/participant.ts`; the planned records in `src/study/plan-participants.ts`. Computer-use and shared-world participants run a `DesktopParticipantRun` (`src/routes/computer-use/types.ts`). The terminal session (`src/routes/terminal/session.ts`), scripted surfaces (`src/routes/scripted/plan.ts`) and fixture simulations (`src/run/dry-run.ts`) still build their own. |
 | persona             | Who a participant is: traits, background, device. `humanish/personas/<id>.yaml`.                                                                                                                                                                                                                                                                                                                                                                                                                                                      | `src/study/persona.ts`, `src/study/persona-resolve.ts`                                                                                                                                                                                                                                                                                                                                                      |
@@ -36,7 +36,7 @@ bundle and output fields keep older spellings; the second table maps them.
 | topology            | Whether participants share one instance of the subject. `route: shared-world` gives one for all; every other route gives each participant its own (`per-lane-worlds`). `run.json` records it as `sharedWorld.topology`.                                                                                                                                                                                                                                                                                                               | `route` in `StudyConfig` (`src/study/types.ts`); `isSharedWorldComposition` in `src/study/routing.ts`                                                                                                                                                                                                                                                                                                       |
 | plane               | The one instance of the subject that a shared-world study's participants use together. A `provisioned` plane is a `clone` or `local-tree` subject that humanish serves in a sandbox, with a declared subject state. An `external-public` plane is a deployment already running at `subject.appUrl`, whose owner the study names in `subject.publicTarget.owner`. Each plane has at least two participants.                                                                                                                            | `SharedWorldPlane` in `src/study/plan-types.ts`; `src/routes/shared-world/`                                                                                                                                                                                                                                                                                                                                 |
 | stream              | One live or recorded view of a participant, such as a browser or a terminal, in `run.json` and the Observer.                                                                                                                                                                                                                                                                                                                                                                                                                          | `RunStream` in `src/run/streams.ts`                                                                                                                                                                                                                                                                                                                                                                         |
-| Observer            | The single-file review page built from `observer/observer-data.json` (`humanish.observer-data.v2`).                                                                                                                                                                                                                                                                                                                                                                                                                                   | `observer/` (the page), `src/observer/` (data, render, serve)                                                                                                                                                                                                                                                                                                                                               |
+| Observer            | The single-file review page built from `observer/observer-data.json` (`humanish.observer-data.v3`).                                                                                                                                                                                                                                                                                                                                                                                                                                   | `observer/` (the page), `src/observer/` (data, render, serve)                                                                                                                                                                                                                                                                                                                                               |
 | evidence            | Files a run keeps: screenshots, traces, transcripts, recordings. `verify` checks them and grades share-safety.                                                                                                                                                                                                                                                                                                                                                                                                                        | `src/evidence/`; `verifyRun` in `src/verify/verify.ts`                                                                                                                                                                                                                                                                                                                                                      |
 | analysis            | An independent model reading of a finished run's retained evidence, kept apart from participant feedback. Supported live runs request it automatically, and `humanish analyze` runs it on demand. It records findings and observations as `humanish.study-analysis.v1`.                                                                                                                                                                                                                                                               | `src/analysis/`; `runAutomaticAnalysis` in `src/analysis/automatic.ts`; `analyzeRun` in `src/analysis/service.ts`; `ANALYSIS_SCHEMA` in `src/analysis/types.ts`                                                                                                                                                                                                                                             |
 | finding             | A result of analysis, with citations to evidence in the run. Each finding groups the observations that support it.                                                                                                                                                                                                                                                                                                                                                                                                                    | `src/analysis/`                                                                                                                                                                                                                                                                                                                                                                                             |

package/README.md

@@ -335,7 +335,7 @@ and the library exports.
 - Breaking changes ship together in one minor release. Its release notes carry a migration
   table: each old form, its new form, and the change to make.
 - The study file and the run bundle carry versioned schemas, `humanish.study.v3` and
-  `humanish.run-bundle.v3`. A breaking change to either one bumps its version.
+  `humanish.run-bundle.v4`. A breaking change to either one bumps its version.
 
 Pin the exact version in `package.json` and read [CHANGELOG.md](CHANGELOG.md) before upgrading.
 

package/dist/actors/computer-use/loop/output-limit.js

@@ -17,5 +17,8 @@ export function retryAfterOutputLimit(session, turn, turnNumber, alreadyRetried)
     session.usage.record(turn, "interaction");
     const output = turn.usage?.output;
     session.trace.record("notice", () => notice("warn", "provider reply cut off by the output limit; asking again", `provider turn ${turnNumber} reached the output-token limit${output === undefined ? "" : ` after ${output} output tokens`} before any action; its actions were not run, and the same request is sent once more`));
+    // haltBeforeTurn checks spend again before the resend. Checking here as well keeps a reply that
+    // crossed a cap out of the trace's narration, as after a complete reply, and keeps a crossed
+    // cap ahead of unknown usage.
     return spendStop(session) ?? unknownSpendStop(session) ?? "retry";
 }

package/dist/actors/computer-use/loop/types.d.ts

@@ -386,19 +386,25 @@ export interface CuaLoopOptions {
      */
     maxUsd?: number;
     /**
-     * Injected pure per-turn cost estimator (keeps the loop free of the operator rate table and
-     * makes the cap deterministic in tests). Given running (input, output) token totals, returns the
-     * estimated USD, or null when unpriceable. Only consulted when `maxUsd` is set. A null estimate
-     * mid-run cannot trip the cap; preflight already guaranteed a rate exists, so a null here is a
-     * vanished-rate harness condition, not a silent uncapped pass.
+     * Injected pure cost estimator (keeps the loop free of the operator rate table and makes the
+     * cap deterministic in tests). Given running (input, output) token totals, returns the
+     * estimated USD, or null when unpriceable. Only consulted when `maxUsd` is set. The loop calls it
+     * after each reply and again before each provider request, the first included, when every total
+     * is 0 and the usage has no `turns`. An estimate of zero usage above `maxUsd` therefore stops the
+     * session before its first request. A null estimate mid-run cannot trip the cap; preflight
+     * already guaranteed a rate exists, so a null here is a vanished-rate harness condition, not a
+     * silent uncapped pass.
      */
     estimateTurnCostUsd?: (usage: ActorTokenUsage) => number | null;
     /**
-     * Run-level spend guard: called with this participant's running usage each turn, at the same
-     * point the per-participant cap is checked. Returns a human-readable reason when the study's shared
-     * budget is exhausted, else null. On a non-null return the loop stops with `budget_reached`
-     * regardless of material progress: a study-level stop is a recruiting decision hitting its
-     * limit, not this participant's runaway, so it never reads as `gave_up`.
+     * Run-level spend guard: called with this participant's running usage at each point the
+     * per-participant cap is checked, after each reply and before each provider request. The first
+     * call comes before the first request, with every total at 0 and no `turns`. How many calls a
+     * turn makes depends on the loop's checks, so a guard should decide from the usage it is given
+     * and the ledger it reads. Returns a human-readable reason when the study's shared budget is
+     * exhausted, else null. On a non-null return the loop stops with `budget_reached` regardless of
+     * material progress: a study-level stop is a recruiting decision hitting its limit, not this
+     * participant's runaway, so it never reads as `gave_up`.
      */
     overRunBudget?: (usage: ActorTokenUsage) => string | null;
     /**

package/dist/actors/computer-use/loop.js

@@ -201,11 +201,16 @@ function refuseAccountBilledCaps(options) {
         });
     }
 }
+/**
+ * The stops checked before every provider request. The spend caps are checked here as well as
+ * after each reply, because a sibling participant can exhaust the study budget while this one acts.
+ */
 function haltBeforeTurn(session, observation) {
     if (session.signal?.aborted)
         return stops.harnessAborted;
     return (clockEndStop(session, observation) ??
-        (session.remaining() < 0 ? stops.timeLimit(session) : undefined));
+        (session.remaining() < 0 ? stops.timeLimit(session) : undefined) ??
+        spendStop(session));
 }
 /**
  * Build the next request; the context hint and safety acknowledgements ride it once. The

package/dist/feedback/export.js

@@ -180,7 +180,8 @@ async function readObserverSlot(cwd, runInput, runRoot, runId, verified, deps, w
  * even if an older saved HTML was never refreshed; old recording-only exports keep the saved slot.
  * A page saved since observer-data v2 holds no participant's items in its slot, and verify does not
  * read the page's compressed copy, so its export is the current projection of the run and its
- * traces. Undefined when run.json no longer matches the analysis, or a v2 page's run cannot be read.
+ * traces, for a page saved as v2 too. Undefined when run.json no longer matches the analysis, or
+ * such a page's run cannot be read.
  */
 async function currentObserverData(runPaths, analysis, slot) {
     const saved = JSON.parse(slot);
@@ -189,7 +190,7 @@ async function currentObserverData(runPaths, analysis, slot) {
         const schema = saved?.schema;
         if (schema === "humanish.observer-data.v1")
             return { value: observerDataFromSaved(saved) };
-        if (schema !== OBSERVER_DATA_SCHEMA)
+        if (schema !== OBSERVER_DATA_SCHEMA && schema !== "humanish.observer-data.v2")
             return { value: saved };
         const bundle = await sourceTraces(runPaths, await source());
         return bundle === undefined ? undefined : { value: buildObserverData(bundle) };

package/dist/observer/data.d.ts

@@ -2,7 +2,12 @@ import { type ActorEnding } from "../actors/stop-cause.js";
 import type { RunBundle, RunCostSummary, RunEvent, RunSimulation } from "../run/bundle.js";
 import { type RunDisplay } from "../run/display.js";
 import type { RunStream, RunStreamKind } from "../run/streams.js";
-export declare const OBSERVER_DATA_SCHEMA = "humanish.observer-data.v2";
+/**
+ * The Observer's data. From v2 observer-data.json leaves out each participant's items, and from v3
+ * each participant's per-request lists (REQUEST_LISTS in src/run/stream-trace.ts), which no part
+ * of the page shows.
+ */
+export declare const OBSERVER_DATA_SCHEMA = "humanish.observer-data.v3";
 interface ObserverArtifactLink {
     label: string;
     href: string;
@@ -10,7 +15,7 @@ interface ObserverArtifactLink {
 }
 export interface ObserverData {
     schema: typeof OBSERVER_DATA_SCHEMA;
-    schemaVersion: 2;
+    schemaVersion: 3;
     generatedAt: string;
     /** Served-only liveness observation. Never a replacement for the recorded evidence verdict. */
     runtime?: {
@@ -124,10 +129,12 @@ interface ObserverLaneGroup {
  */
 export declare function buildObserverData(bundle: RunBundle, generatedAt?: string, statusRecord?: unknown): ObserverData;
 /**
- * A saved Observer projection as this version reads it, unchecked as before: an observer-data.json
- * written before v2 holds every participant's items. A v2 one holds none, so it is read only when
- * no participant has any: showing participants with no steps would say they did nothing, and the
- * items are in the run's traces. Null for a v2 projection that leaves items out.
+ * A saved Observer projection as this version reads it, as v3 and unchecked as before. An
+ * observer-data.json written before v2 holds every participant's items, and its per-request lists,
+ * which are left out. A v2 or v3 one holds no items, so it is read only when no participant has
+ * any: showing participants with no steps would say they did nothing, and the items are in the
+ * run's traces. Null for a projection that leaves items out, and for a schema other than v1, v2
+ * and v3.
  */
 export declare function observerDataFromSaved(value: unknown): ObserverData | null;
 /** Refresh presentation from recorded actor evidence, including older exported snapshots. */

package/dist/observer/data.js

@@ -3,8 +3,15 @@ import { isCommsReceivingEvidence, receivingAnalysisContext } from "../comms/rec
 import { formatParticipantOutcomes, formatRunTaskFunnel, participantOutcomeDetails, withCuaReviewProvenance, } from "../run/outcomes.js";
 import { cuaGoalSource, CUA_COMPLETION_NOTE } from "../actors/goal-source.js";
 import { bundleDisplayFacts, runDisplay } from "../run/display.js";
+import { withoutRequestLists } from "../run/stream-trace.js";
+import { isRecord } from "../run/type-guards.js";
 import { joinObserverData, shellNeedsItems } from "./participant-items.js";
-export const OBSERVER_DATA_SCHEMA = "humanish.observer-data.v2";
+/**
+ * The Observer's data. From v2 observer-data.json leaves out each participant's items, and from v3
+ * each participant's per-request lists (REQUEST_LISTS in src/run/stream-trace.ts), which no part
+ * of the page shows.
+ */
+export const OBSERVER_DATA_SCHEMA = "humanish.observer-data.v3";
 /** Discard a forged or stale runtime grant before projecting persisted evidence. */
 export function recordedStreamEmbed(embed) {
     if (!embed)
@@ -52,7 +59,7 @@ export function buildObserverData(bundle, generatedAt = new Date().toISOString()
                 : stream.status);
         byKind[stream.kind] += 1;
         return {
-            ...stream,
+            ...withoutStreamLists(stream),
             ...(stream.embed === undefined ? {} : { embed: recordedStreamEmbed(stream.embed) }),
             status,
             sim,
@@ -81,7 +88,7 @@ export function buildObserverData(bundle, generatedAt = new Date().toISOString()
     const active = streams.filter((stream) => stream.status === "running" || stream.status === "preparing").length;
     return withObserverEndings({
         schema: OBSERVER_DATA_SCHEMA,
-        schemaVersion: 2,
+        schemaVersion: 3,
         generatedAt,
         run: {
             runId: bundle.runId,
@@ -142,19 +149,42 @@ export function buildObserverData(bundle, generatedAt = new Date().toISOString()
         },
     });
 }
+/** The stream without its participant's per-request lists; a saved stream that is not an object as it is. */
+function withoutStreamLists(stream) {
+    if (!isRecord(stream))
+        return stream;
+    return {
+        ...stream,
+        ...(isRecord(stream.actor) ? { actor: withoutRequestLists(stream.actor) } : {}),
+        ...(isRecord(stream.liveActor) ? { liveActor: withoutRequestLists(stream.liveActor) } : {}),
+    };
+}
 /**
- * A saved Observer projection as this version reads it, unchecked as before: an observer-data.json
- * written before v2 holds every participant's items. A v2 one holds none, so it is read only when
- * no participant has any: showing participants with no steps would say they did nothing, and the
- * items are in the run's traces. Null for a v2 projection that leaves items out.
+ * A saved Observer projection as this version reads it, as v3 and unchecked as before. An
+ * observer-data.json written before v2 holds every participant's items, and its per-request lists,
+ * which are left out. A v2 or v3 one holds no items, so it is read only when no participant has
+ * any: showing participants with no steps would say they did nothing, and the items are in the
+ * run's traces. Null for a projection that leaves items out, and for a schema other than v1, v2
+ * and v3.
  */
 export function observerDataFromSaved(value) {
     const saved = value;
-    if (saved.schema === "humanish.observer-data.v1")
-        return { ...saved, schema: OBSERVER_DATA_SCHEMA, schemaVersion: 2 };
+    const schema = saved.schema;
+    if (schema === "humanish.observer-data.v1") {
+        const streams = Array.isArray(saved.streams)
+            ? saved.streams.map(withoutStreamLists)
+            : saved.streams;
+        return { ...saved, streams, schema: OBSERVER_DATA_SCHEMA, schemaVersion: 3 };
+    }
+    if (schema !== "humanish.observer-data.v2" && schema !== OBSERVER_DATA_SCHEMA)
+        return null;
     return shellNeedsItems(saved)
         ? null
-        : joinObserverData(saved, null);
+        : {
+            ...joinObserverData(saved, null),
+            schema: OBSERVER_DATA_SCHEMA,
+            schemaVersion: 3,
+        };
 }
 const GOAL_SOURCE_LABELS = {
     participant_report: "Reported complete",