humanish 0.127.1 → 0.129.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +26 -0
- package/CONTEXT.md +2 -2
- package/README.md +1 -1
- package/dist/actors/computer-use/loop/output-limit.js +3 -0
- package/dist/actors/computer-use/loop/types.d.ts +16 -10
- package/dist/actors/computer-use/loop.js +6 -1
- package/dist/analysis/evidence.js +1 -4
- package/dist/analysis/validation.js +1 -1
- package/dist/cli/commands/study-format.js +2 -1
- package/dist/feedback/export.js +3 -2
- package/dist/observer/data.d.ts +13 -6
- package/dist/observer/data.js +40 -10
- package/dist/observer/feed-cursors.d.ts +3 -1
- package/dist/observer/feed-cursors.js +5 -2
- package/dist/observer/run-routes.js +40 -7
- package/dist/observer-app.html +8 -8
- package/dist/routes/computer-use/fanout-bundle.js +4 -7
- package/dist/routes/computer-use/participant-model.js +1 -1
- package/dist/routes/shared-world/bundle-records.js +6 -1
- package/dist/routes/shared-world/bundle.d.ts +8 -0
- package/dist/routes/shared-world/bundle.js +33 -13
- package/dist/routes/shared-world/participant-specs.d.ts +2 -1
- package/dist/routes/shared-world/participant-specs.js +5 -2
- package/dist/routes/shared-world/result.js +13 -7
- package/dist/routes/shared-world/types.d.ts +2 -0
- package/dist/run/bundle.d.ts +7 -4
- package/dist/run/bundle.js +10 -4
- package/dist/run/detail.js +8 -2
- package/dist/run/locate.d.ts +2 -2
- package/dist/run/locate.js +2 -2
- package/dist/run/outcomes.d.ts +7 -0
- package/dist/run/outcomes.js +14 -0
- package/dist/run/run.js +11 -3
- package/dist/run/stream-trace.d.ts +41 -14
- package/dist/run/stream-trace.js +195 -42
- package/dist/run/streams.d.ts +4 -1
- package/dist/study/bundle-estimate.js +17 -17
- package/dist/verify/costs.d.ts +7 -3
- package/dist/verify/costs.js +27 -12
- package/dist/verify/verify.js +14 -8
- package/docs/architecture/github-feedback-loop.md +1 -1
- package/docs/architecture/observer.md +32 -10
- package/docs/contracts/README.md +1 -1
- package/docs/contracts/run-bundle.md +102 -18
- package/docs/contracts/schemas.md +55 -45
- package/docs/contracts/study-analysis.md +1 -1
- package/docs/decisions/0002-observer-is-one-self-contained-file.md +1 -1
- package/package.json +1 -1
|
@@ -129,7 +129,7 @@ humanish_swarm:
|
|
|
129
129
|
- .env*
|
|
130
130
|
- .github/workflows/**
|
|
131
131
|
- infra/**
|
|
132
|
-
artifact_schema_version: humanish.run-bundle.
|
|
132
|
+
artifact_schema_version: humanish.run-bundle.v4
|
|
133
133
|
credential_manifest: []
|
|
134
134
|
network_policy: no_network
|
|
135
135
|
spend_policy: no_spend
|
|
@@ -25,12 +25,17 @@ Observer commands may add derived artifacts:
|
|
|
25
25
|
```
|
|
26
26
|
|
|
27
27
|
`run.json` remains the source bundle. `observer/observer-data.json`
|
|
28
|
-
(`humanish.observer-data.
|
|
29
|
-
with every participant except their trace items
|
|
30
|
-
artifact list
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
28
|
+
(`humanish.observer-data.v3`) is the normalized view model consumed by the Observer,
|
|
29
|
+
with every participant except their trace items, the screenshot entries of their
|
|
30
|
+
artifact list, and their per-request lists (`tokenUsage.turns`, `conversation.requests`
|
|
31
|
+
and `providerRequests`, `REQUEST_LISTS` in `src/run/stream-trace.ts`), which no view
|
|
32
|
+
shows. A participant working for hours records thousands of items and a screenshot each
|
|
33
|
+
turn and makes a model request every few seconds, so these are most of a long run's
|
|
34
|
+
bytes; apart, the saved file stays small enough for `verify` to read, and a flush does
|
|
35
|
+
not rewrite them. The items and lists stay in the run's traces (`actors/<stream>.json`).
|
|
36
|
+
`buildObserverData` leaves the lists out of every projection: observer-data.json, the
|
|
37
|
+
page's data slot, the served feed and the HTML export. The page's validator admits an
|
|
38
|
+
account participant's `executionProfile` without its receipts. `events.ndjson` is the appendable event stream
|
|
34
39
|
contract that live adapters update while a run is active.
|
|
35
40
|
|
|
36
41
|
`observer/index.html` carries both parts: the data slot holds observer-data.json's
|
|
@@ -41,9 +46,13 @@ trace items compress about eight times. The page joins the two before it renders
|
|
|
41
46
|
(`joinObserverData` in `src/observer/participant-items.ts`), so every view reads the
|
|
42
47
|
same data as before. A stream whose screenshot entries moved records how many in
|
|
43
48
|
`screenshotArtifacts`, and the page renders nothing from items that are another
|
|
44
|
-
run's or miss a participant's items or screenshot entries. The
|
|
45
|
-
|
|
46
|
-
|
|
49
|
+
run's or miss a participant's items or screenshot entries. The page reads only v3
|
|
50
|
+
data: it checks the schema in the data slot and the feed before it validates
|
|
51
|
+
(`observer/lib/data.ts`). A saved observer-data.json is read as v3
|
|
52
|
+
(`observerDataFromSaved`): a v1 file, which holds the items, without its lists, and
|
|
53
|
+
a v2 or v3 file only when no participant has items to join; the server answers one
|
|
54
|
+
that leaves items out, or one of another schema, as unavailable. `humanish export` rebuilds a page saved as v2
|
|
55
|
+
from the run and its traces, as it rebuilds a current one.
|
|
47
56
|
|
|
48
57
|
## Stream Model
|
|
49
58
|
|
|
@@ -88,6 +97,19 @@ A page that cannot join a body sends no cursors with its next poll. Static
|
|
|
88
97
|
operator path. Agents and CI should use `humanish watch --json --no-open` for
|
|
89
98
|
the same fresh evidence without browser open or a long-running process.
|
|
90
99
|
|
|
100
|
+
The feed reads no live file whose index passes the 32 MiB read limit
|
|
101
|
+
(`RUN_ARTIFACT_MAX_BYTES`); a request for the file by its path still gets the whole file. A page
|
|
102
|
+
that sends a cursor for it keeps the items it holds: the
|
|
103
|
+
server answers with no items past the cursor, gives the page's cursor back as `live`, and adds a
|
|
104
|
+
`live-items.not-read` warning to the participant's timeline, so later polls send the same cursor
|
|
105
|
+
and get the same answer. A poll that sends no cursor for that file gets the participant's trace
|
|
106
|
+
as unavailable, with the file's size and the limit, and the page drops any items it held. Such a
|
|
107
|
+
poll comes from a page opened after the file passed the limit or after the writer started a new
|
|
108
|
+
generation, from the poll after a body the page could not join, and from a page whose cursors
|
|
109
|
+
pass 8,000 characters (`MAX_AFTER_CHARS` in `observer/lib/live.ts`), which sends none. When the
|
|
110
|
+
run finishes, a trace file past the limit is refused too, and every page shows that trace as
|
|
111
|
+
unavailable.
|
|
112
|
+
|
|
91
113
|
### Reopening a run
|
|
92
114
|
|
|
93
115
|
The TUI's **Open in Observer** action opens an HTTP view of the selected run's saved
|
|
@@ -281,7 +303,7 @@ These remain adapter work:
|
|
|
281
303
|
## The rendering layer
|
|
282
304
|
|
|
283
305
|
The renderer is the `observer/` workspace: a Vite single-file build on the
|
|
284
|
-
`@humanish` registry tokens, frozen against `humanish.observer-data.
|
|
306
|
+
`@humanish` registry tokens, frozen against `humanish.observer-data.v3`
|
|
285
307
|
(`tests/observer/data-contract.test.ts`). The root build copies the workspace
|
|
286
308
|
artifact to `dist/observer-app.html`; in a repo checkout a missing or stale
|
|
287
309
|
artifact auto-builds, and an unconditional preflight at CLI startup makes a
|
package/docs/contracts/README.md
CHANGED
|
@@ -8,7 +8,7 @@ disagree.
|
|
|
8
8
|
(`RunStudyOptions`) and the exports removed in 0.107.0.
|
|
9
9
|
- [core.md](core.md): the records a run writes under `.humanish/runs/<id>/` and how each route
|
|
10
10
|
starts a run.
|
|
11
|
-
- [run-bundle.md](run-bundle.md): `humanish.run-bundle.
|
|
11
|
+
- [run-bundle.md](run-bundle.md): `humanish.run-bundle.v4`, the evidence contract that `verify`
|
|
12
12
|
checks.
|
|
13
13
|
- [policy.md](policy.md): credential, network, spend, redaction, sharing and assisted-run
|
|
14
14
|
boundaries.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Run Bundle Contract
|
|
2
2
|
|
|
3
|
-
`humanish.run-bundle.
|
|
3
|
+
`humanish.run-bundle.v4` is the evidence contract. The TypeScript shape in `src/run/bundle.ts`
|
|
4
4
|
(with `streams[]` in `src/run/streams.ts` and `sharedWorld` as `SharedWorldEvidence` in
|
|
5
5
|
`src/run/shared-world-evidence.ts`) and the fail-closed checks in `src/verify/verify.ts` are the
|
|
6
6
|
source of truth. This page explains the stable public fields and the extension rules; it does not
|
|
@@ -39,7 +39,8 @@ trace:
|
|
|
39
39
|
|
|
40
40
|
run.json keeps every field readers list runs by: each stream's status, judged status,
|
|
41
41
|
assignment and geometry, and the actor's status, completion reason, counts, persona, debrief,
|
|
42
|
-
impressions, token usage and estimated cost. Each trace is written once.
|
|
42
|
+
impressions, token usage totals and estimated cost. Each trace is written once. From v4 the
|
|
43
|
+
per-request lists are only in the trace file ([Per-request lists](#per-request-lists)).
|
|
43
44
|
|
|
44
45
|
`trace.path` must be one of those trace file names (`actor.json`, `actors/<name>.json`,
|
|
45
46
|
`actor-<name>.json`, with `<name>` a plain file name); `isStreamTracePath` in
|
|
@@ -48,12 +49,17 @@ it. Any other path, such as `observer/index.html`, fails the shape check.
|
|
|
48
49
|
|
|
49
50
|
A reader outside humanish loads a run through the package's `loadRunBundle(cwd, run)`, with `run`
|
|
50
51
|
a run id or `latest`: it returns run.json with every trace file and live file joined and checked
|
|
51
|
-
as below, for v1, v2 and
|
|
52
|
+
as below, for v1, v2, v3 and v4 bundles, with each participant's per-request lists, and throws an Error named `StreamTraceRefusedError` for a
|
|
52
53
|
file that does not match its index. Inside humanish, readers load a stream through
|
|
53
54
|
`loadStreamTrace(runPaths, stream)`. A v1 stream returns as it is.
|
|
54
55
|
An indexed stream's file is read under the 32 MiB per-file limit (analysis uses its 16 MiB
|
|
55
56
|
source limit for each trace) and refused unless its size, sha256, item count and screenshot count
|
|
56
|
-
match the index. A trace file
|
|
57
|
+
match the index. A trace or live file whose index records more than the limit is refused before it
|
|
58
|
+
is read, and the refusal says the file is larger than humanish reads of one file and that
|
|
59
|
+
restoring a copy does not help. A trace file whose index is under the limit and whose bytes are
|
|
60
|
+
over it changed after the run wrote it, and its refusal says to restore the run folder from a copy.
|
|
61
|
+
A live file may hold bytes past its index while the writer appends; the loader reads only the
|
|
62
|
+
indexed bytes. A trace file belongs to one participant: every stream whose index names a file
|
|
57
63
|
another stream's index names, compared without case as a case-insensitive file system compares
|
|
58
64
|
names, is refused before any file is read. The writer claims names the same way. In a v2 run.json every `actor`
|
|
59
65
|
must have an index: an actor with no index, with
|
|
@@ -82,8 +88,8 @@ stream with a `liveActor`:
|
|
|
82
88
|
`streams[]` (from 1). Each line is one trace item as JSON, scrubbed of the run's raw sandbox ids
|
|
83
89
|
as it is appended. A live file is never truncated or rewritten;
|
|
84
90
|
- in run.json, `liveActor` as `humanish.live-actor.v2`: its summary fields (`updatedAt`,
|
|
85
|
-
`persona`, `tokenUsage`, `ids`, `executionProfile`, `
|
|
86
|
-
`
|
|
91
|
+
`persona`, `tokenUsage`, `ids`, `executionProfile`, `historyTurnsOmitted`, `estimatedCost`, and
|
|
92
|
+
in v3 `providerRequests`) and, in place of `items`, the index of the prefix written so far:
|
|
87
93
|
|
|
88
94
|
```yaml
|
|
89
95
|
liveActor:
|
|
@@ -122,7 +128,7 @@ Readers load live items through the same loader as traces: `loadStreamTraces` re
|
|
|
122
128
|
indexed bytes of the live file (bytes appended after run.json was written are not read), refuses
|
|
123
129
|
them unless their size, sha256 and line count match the index and each line is a JSON object,
|
|
124
130
|
and returns the stream with `liveActor.items` and without `itemsFile`. It refuses a `liveActor`
|
|
125
|
-
schema it does not know, and inline live items (`humanish.live-actor.v1`) in a v3 run.json. A
|
|
131
|
+
schema it does not know, and inline live items (`humanish.live-actor.v1`) in a v3 or v4 run.json. A
|
|
126
132
|
reader that cannot read a live file reads run.json again once and loads that bundle instead, since
|
|
127
133
|
the writer may have removed a superseded generation, or every live file at finish, after the
|
|
128
134
|
reader read run.json. v1 and v2 run.json keep `liveActor.items` inline (`humanish.live-actor.v1`)
|
|
@@ -132,21 +138,99 @@ Readers that show a run (the Observer server, run detail and the TUI, the review
|
|
|
132
138
|
provenance) load traces through the same loader and rules. A participant whose trace the loader
|
|
133
139
|
refused shows none of the items run.json or a file supplied for it: its stream carries
|
|
134
140
|
`traceUnavailable` with the reason, its Observer timeline a `trace.unavailable` warning, and its
|
|
135
|
-
run-detail entry `evidenceUnavailable`. The Observer server reads the saved
|
|
136
|
-
|
|
141
|
+
run-detail entry `evidenceUnavailable`. The Observer server reads run.json, and the saved
|
|
142
|
+
observer-data.json, under the same 32 MiB limit; past it the server answers 404 with a reason
|
|
143
|
+
naming the limit. It reads the saved observer-data.json only while a run has no run.json at all; a run.json that is there and cannot be read, parsed or
|
|
137
144
|
used as a bundle answers 404 with the reason, never the saved projection. In safe serving
|
|
138
145
|
(`observe --all --safe`) every read is checked against the files admitted with the run's
|
|
139
146
|
share_ready verification, so a run.json changed after admission is refused and the admitted
|
|
140
147
|
projection is served.
|
|
141
|
-
observer-data.json is built from the loaded bundle. From 0.124.0 it
|
|
142
|
-
|
|
143
|
-
|
|
148
|
+
observer-data.json is built from the loaded bundle. From 0.124.0 it leaves out each participant's
|
|
149
|
+
trace items, which stay in the traces, and from this release it is `humanish.observer-data.v3`,
|
|
150
|
+
which also leaves out each participant's per-request lists (docs/architecture/observer.md).
|
|
151
|
+
|
|
152
|
+
## Per-request lists
|
|
153
|
+
|
|
154
|
+
From v4, three lists that gain one entry per model request are only in each participant's trace
|
|
155
|
+
file: `tokenUsage.turns` (each request's usage), `conversation.requests` (each request's mode and,
|
|
156
|
+
in explicit_context, what it carried) and `providerRequests` (an account participant's receipt for
|
|
157
|
+
each dispatch). Before v4 run.json held a copy of each beside the trace file's, and a flush wrote
|
|
158
|
+
every running participant's usage rows into run.json and observer-data.json. A synthetic run of
|
|
159
|
+
30 participants making 2,880 requests each, written through the run writer, leaves a run.json of
|
|
160
|
+
about 0.1 MB, where the same run with the lists was 16 MB with threaded conversations and 42 MB
|
|
161
|
+
with account receipts. `REQUEST_LISTS` in `src/run/stream-trace.ts` names the three paths, and the
|
|
162
|
+
writer, the loader and the Observer projection read it.
|
|
163
|
+
|
|
164
|
+
The writer (`splitStreamTraces`) writes a finished participant's trace file as before, lists
|
|
165
|
+
included, and run.json's `streams[].actor` without `items` and without the three lists:
|
|
166
|
+
`tokenUsage` keeps its totals and `conversation` its `mode` and `summarizedTurns`.
|
|
167
|
+
|
|
168
|
+
For a running participant, run.json's `liveActor` holds no `tokenUsage.turns` and no
|
|
169
|
+
`providerRequests`. When it has `tokenUsage` and a model id in `ids.model`, the writer prices its
|
|
170
|
+
rows before leaving them out and records the figure as `liveActor.estimatedCost`
|
|
171
|
+
(`humanish.actor-estimated-cost.v1`, from `estimateActorCostForExecution` in
|
|
172
|
+
`src/run/pricing.ts`), in place of any estimate the producer passed. The long-context tier prices
|
|
173
|
+
a request by its own input size, so the totals alone can price a long session at under half its
|
|
174
|
+
cost. An account participant's estimate is a null figure with the reason
|
|
175
|
+
`account_billing_unknown`.
|
|
144
176
|
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
177
|
+
```yaml
|
|
178
|
+
liveActor:
|
|
179
|
+
schema: humanish.live-actor.v2
|
|
180
|
+
updatedAt: "<ISO timestamp>"
|
|
181
|
+
tokenUsage: { input: 412000, output: 9100, cachedInput: 380000, cacheWriteInput: 21000 }
|
|
182
|
+
ids: { model: gpt-5.6-sol }
|
|
183
|
+
estimatedCost:
|
|
184
|
+
schema: humanish.actor-estimated-cost.v1
|
|
185
|
+
estimatedCostUsd: 0.4123
|
|
186
|
+
ratesAsOf: "2026-09-03"
|
|
187
|
+
source: "<rate source>"
|
|
188
|
+
itemsFile: { path: live/1.1.ndjson, generation: 1, items: 4210, bytes: 1048211, sha256: "<hex>" }
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
The rows and receipts are written once, in the trace file, when the participant finishes. A run
|
|
192
|
+
interrupted before then keeps each running participant's totals, estimate and live file. No file
|
|
193
|
+
holds that participant's rows or receipts, including a receipt with `usageComplete: false` or
|
|
194
|
+
`cleanup: "unconfirmed"`, so neither the trace path nor `loadRunBundle` recovers them.
|
|
195
|
+
|
|
196
|
+
The loader joins the lists back for every indexed stream, whatever its version: each list the
|
|
197
|
+
trace file holds goes into run.json's `tokenUsage` or `conversation` when run.json has that
|
|
198
|
+
object, and `providerRequests` at the actor's top level. A v2 or v3 writer wrote run.json's copy
|
|
199
|
+
and the file's from one actor, so their result does not change. A trace file whose `conversation`
|
|
200
|
+
has no `requests` list, or that holds one of the three as something other than a list, is
|
|
201
|
+
refused.
|
|
202
|
+
|
|
203
|
+
Which writer rules the loader applies depends on the version run.json declares. A run.json that
|
|
204
|
+
breaks one did not come from that version's writer, and the stream is refused before any file is
|
|
205
|
+
read:
|
|
206
|
+
|
|
207
|
+
| Rule | v1 | v2 | v3 | v4 |
|
|
208
|
+
| ----------------------------------------------------------- | ------- | ------- | ------- | ------- |
|
|
209
|
+
| Two streams index one trace file | refused | refused | refused | refused |
|
|
210
|
+
| An actor without a trace index | read | refused | refused | refused |
|
|
211
|
+
| `liveActor` in a run whose `outcome.state` is `finished` | read | refused | refused | refused |
|
|
212
|
+
| Inline live items (`humanish.live-actor.v1`) | read | read | refused | refused |
|
|
213
|
+
| One of the three lists in run.json's `actor` or `liveActor` | read | read | read | refused |
|
|
214
|
+
|
|
215
|
+
A reader that shows a run (`displayedTraces`) shows a refused stream without the lists run.json held
|
|
216
|
+
for it, so run detail never prices rows the loader refused. Run detail shows a refused participant's
|
|
217
|
+
cost only when run.json records an `estimatedCost` for it, since its totals alone could price under
|
|
218
|
+
half of what its rows give. verify checks a running participant's `liveActor.estimatedCost` as it
|
|
219
|
+
checks a finished one's: its schema, a `ratesAsOf` date and `source` beside a figure, and a reason
|
|
220
|
+
beside a null figure. In a v4 run.json it no longer asks a running account participant for receipts;
|
|
221
|
+
the check that account dollars stay unknown still applies, and the run still grades `local_only`
|
|
222
|
+
with `RUN_NOT_FINISHED`. verify does not check the receipts of a participant whose trace the loader
|
|
223
|
+
refused, in any version: the refusal fails `local evidence artifacts exist`, and from v4 the
|
|
224
|
+
receipts are only in that trace.
|
|
225
|
+
|
|
226
|
+
Compatibility: humanish 0.122.0 and earlier read only v1 and refuse a later run.json by its schema
|
|
227
|
+
(`run schema` fails in verify, analysis refuses the source); 0.123.0 and 0.124.0 read v1 and v2
|
|
228
|
+
and refuse v3 and v4 the same way. 0.125.0 and 0.126.0 read v1, v2 and v3: their verify fails
|
|
229
|
+
`run schema` on a v4 run.json, their analysis refuses it, and their `loadRunBundle` returns null.
|
|
230
|
+
Their Observer server and run detail parse run.json without checking its schema and show a v4
|
|
231
|
+
run without the lists; the 0.126.0 page refuses an account participant's data without receipts.
|
|
232
|
+
This release reads v1, v2 and v3 bundles as they are, with no migration. An exported copy
|
|
233
|
+
(`humanish export --format bundle`) is written as v4 whatever the source version.
|
|
150
234
|
|
|
151
235
|
## Purpose
|
|
152
236
|
|
|
@@ -157,7 +241,7 @@ feedback drafts and future public issues.
|
|
|
157
241
|
## Minimum Bundle Shape
|
|
158
242
|
|
|
159
243
|
```yaml
|
|
160
|
-
schema: humanish.run-bundle.
|
|
244
|
+
schema: humanish.run-bundle.v4 # v3, v2 and v1 bundles are still read
|
|
161
245
|
runId: "<core run id>"
|
|
162
246
|
mode: "dry-run|live"
|
|
163
247
|
simCount: 1
|
|
@@ -24,38 +24,38 @@ workflow without leaking private upstream truth into core.
|
|
|
24
24
|
|
|
25
25
|
## Contract Index
|
|
26
26
|
|
|
27
|
-
| Contract | Schema
|
|
28
|
-
| --------------------------------- |
|
|
29
|
-
| Run bundle | `humanish.run-bundle.
|
|
30
|
-
| Adapter | `humanish.adapter.v1`
|
|
31
|
-
| Study file | `humanish.study.v3`
|
|
32
|
-
| Persona | `humanish.persona.v1`
|
|
33
|
-
| Scenario | `humanish.scenario.v1`
|
|
34
|
-
| Actor trace | `humanish.actor-trace.v1`
|
|
35
|
-
| Substrate | reserved (never shipped)
|
|
36
|
-
| Evidence stream | reserved (streams live inside the run bundle)
|
|
37
|
-
| Review | `humanish.review.v1`
|
|
38
|
-
| Study analysis | `humanish.study-analysis.v1` (prompt `study-evidence-7` adds `findings[].headline`, `findings[].experience` and `designFindings[]`; older artifacts omit them; `study-evidence-8` adds participant impressions to the instructions, with no field change)
|
|
39
|
-
| Study analysis correction | `humanish.study-analysis-correction.v1`
|
|
40
|
-
| Analysis findings view | `humanish.analysis-findings.v1` (`review --json` field `analysis`; read-only projection of the selected analysis)
|
|
41
|
-
| Analysis execution receipt | `humanish.analysis-execution.v1`
|
|
42
|
-
| Analysis execution start | `humanish.analysis-execution-start.v1`
|
|
43
|
-
| Verification | `humanish.verify-result.v1`
|
|
44
|
-
| Policy | `humanish.policy.v1` (fixture-only; not engine-validated)
|
|
45
|
-
| Feedback | `humanish.feedback.v1`
|
|
46
|
-
| Terminal cost ledger | `humanish.terminal-cost-ledger.v1`
|
|
47
|
-
| Terminal no-spend proof | `humanish.terminal-no-spend-proof.v1`
|
|
48
|
-
| Pricing (operator-editable rates) | `humanish.pricing.v1` (`src/run/pricing.ts`; dated per-model + E2B desktop rates)
|
|
49
|
-
| Run cost summary | `humanish.run-cost-summary.v1` (additive `RunBundle.cost`; estimate, never a charge)
|
|
50
|
-
| Estimated actor cost | `humanish.actor-estimated-cost.v1` (additive `ActorTrace.estimatedCost`)
|
|
51
|
-
| Model settings | `humanish.model-settings.v1` (additive `ActorTrace.modelSettings`; the reasoning effort the request carried)
|
|
52
|
-
| Affordance use | `humanish.affordance-use.v1` (additive `ActorTrace.affordanceUse`; per-class counts of the routes an actor took)
|
|
53
|
-
| Adapter score | `humanish.adapter-score.v1` (`RunBundle.adapterScore`; namespaced; route-specific acceptance semantics)
|
|
54
|
-
| Adapter artifact | `humanish.adapter-artifact.v1` (`RunBundle.adapterArtifacts[]`; namespaced; local relative proof references)
|
|
55
|
-
| Shared-world evidence | `humanish.shared-world.v1` (additive `RunBundle.sharedWorld` + `RunBundle.attributionClass`; `topologyMode: sequential \| concurrent`)
|
|
56
|
-
| Comms thread | `humanish.comms-thread.v1` (off-app email/SMS the app sent, captured; a `kind: log` run-dir artifact of digests only: from/to/subject/link digests + one OTP count, never raw)
|
|
57
|
-
| Serve result | `humanish.serve-result.v1` (`src/observer/serve.ts` is authoritative)
|
|
58
|
-
| Serve control plane | reserved (`/_humanish/api/*` answers `501` `HUMANISH_SERVE_CONTROL_PLANE_DISABLED` in v1)
|
|
27
|
+
| Contract | Schema | Public-safe fixture |
|
|
28
|
+
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- |
|
|
29
|
+
| Run bundle | `humanish.run-bundle.v4` (each participant's trace in its own file, indexed by `streams[].trace`, with its per-request lists only there; a running participant's items so far in a live file indexed by `streams[].liveActor.itemsFile`, and its totals and `estimatedCost` in run.json; v3 bundles, with the lists in run.json too, v2 bundles, with running items inline, and v1 bundles, with traces inline, are still read) | `synthetic-run-bundle` |
|
|
30
|
+
| Adapter | `humanish.adapter.v1` | `synthetic-cli-adapter` |
|
|
31
|
+
| Study file | `humanish.study.v3` | `first-run` |
|
|
32
|
+
| Persona | `humanish.persona.v1` | `synthetic-maintainer` |
|
|
33
|
+
| Scenario | `humanish.scenario.v1` | `first-run-smoke` |
|
|
34
|
+
| Actor trace | `humanish.actor-trace.v1` | `synthetic-actor-trace` |
|
|
35
|
+
| Substrate | reserved (never shipped) | none |
|
|
36
|
+
| Evidence stream | reserved (streams live inside the run bundle) | see [`run-bundle.md`](run-bundle.md) |
|
|
37
|
+
| Review | `humanish.review.v1` | `contract-proof-review` |
|
|
38
|
+
| Study analysis | `humanish.study-analysis.v1` (prompt `study-evidence-7` adds `findings[].headline`, `findings[].experience` and `designFindings[]`; older artifacts omit them; `study-evidence-8` adds participant impressions to the instructions, with no field change) | see [study analysis](study-analysis.md) and synthetic analysis fixtures in `tests/` |
|
|
39
|
+
| Study analysis correction | `humanish.study-analysis-correction.v1` | see [study analysis](study-analysis.md#human-review-and-sharing) |
|
|
40
|
+
| Analysis findings view | `humanish.analysis-findings.v1` (`review --json` field `analysis`; read-only projection of the selected analysis) | see [study analysis](study-analysis.md#reading-findings) |
|
|
41
|
+
| Analysis execution receipt | `humanish.analysis-execution.v1` | see [study analysis](study-analysis.md#durable-records) |
|
|
42
|
+
| Analysis execution start | `humanish.analysis-execution-start.v1` | see [study analysis](study-analysis.md#durable-records) |
|
|
43
|
+
| Verification | `humanish.verify-result.v1` | `five-check-verify` |
|
|
44
|
+
| Policy | `humanish.policy.v1` (fixture-only; not engine-validated) | `public-safety-policy` |
|
|
45
|
+
| Feedback | `humanish.feedback.v1` | `public-safe-feedback` |
|
|
46
|
+
| Terminal cost ledger | `humanish.terminal-cost-ledger.v1` | see Terminal Cost Ledger below |
|
|
47
|
+
| Terminal no-spend proof | `humanish.terminal-no-spend-proof.v1` | see Terminal Cost Ledger below |
|
|
48
|
+
| Pricing (operator-editable rates) | `humanish.pricing.v1` (`src/run/pricing.ts`; dated per-model + E2B desktop rates) | see Run Cost Summary And Estimated Actor Cost below |
|
|
49
|
+
| Run cost summary | `humanish.run-cost-summary.v1` (additive `RunBundle.cost`; estimate, never a charge) | see Run Cost Summary And Estimated Actor Cost below |
|
|
50
|
+
| Estimated actor cost | `humanish.actor-estimated-cost.v1` (additive `ActorTrace.estimatedCost`) | see Run Cost Summary And Estimated Actor Cost below |
|
|
51
|
+
| Model settings | `humanish.model-settings.v1` (additive `ActorTrace.modelSettings`; the reasoning effort the request carried) | see Actor Trace below |
|
|
52
|
+
| Affordance use | `humanish.affordance-use.v1` (additive `ActorTrace.affordanceUse`; per-class counts of the routes an actor took) | see Affordance Use below |
|
|
53
|
+
| Adapter score | `humanish.adapter-score.v1` (`RunBundle.adapterScore`; namespaced; route-specific acceptance semantics) | see Product-Adapter Extension Seam below |
|
|
54
|
+
| Adapter artifact | `humanish.adapter-artifact.v1` (`RunBundle.adapterArtifacts[]`; namespaced; local relative proof references) | see Product-Adapter Extension Seam below |
|
|
55
|
+
| Shared-world evidence | `humanish.shared-world.v1` (additive `RunBundle.sharedWorld` + `RunBundle.attributionClass`; `topologyMode: sequential \| concurrent`) | see Shared-World Evidence below |
|
|
56
|
+
| Comms thread | `humanish.comms-thread.v1` (off-app email/SMS the app sent, captured; a `kind: log` run-dir artifact of digests only: from/to/subject/link digests + one OTP count, never raw) | see `comms` under Study File |
|
|
57
|
+
| Serve result | `humanish.serve-result.v1` (`src/observer/serve.ts` is authoritative) | none (command result envelope; see Serve Result below) |
|
|
58
|
+
| Serve control plane | reserved (`/_humanish/api/*` answers `501` `HUMANISH_SERVE_CONTROL_PLANE_DISABLED` in v1) | none |
|
|
59
59
|
|
|
60
60
|
## Study File
|
|
61
61
|
|
|
@@ -523,7 +523,7 @@ A study's result has one schema on every route, `humanish.study-result.v1`
|
|
|
523
523
|
(`src/run/study-result.ts`), with `route` and `studyId`. 0.109.0 removed `labId`, which 0.108.0
|
|
524
524
|
wrote with the same value. Route-specific fields stay on their route's result. `humanish run`
|
|
525
525
|
with no study, `observe`, and a refusal made before a route is picked keep
|
|
526
|
-
`humanish.run-result.v1`. The evidence record stays `humanish.run-bundle.
|
|
526
|
+
`humanish.run-result.v1`. The evidence record stays `humanish.run-bundle.v4` in every case.
|
|
527
527
|
The computer-use result carries `plan` (the pre-flight participant table: concurrency, waves,
|
|
528
528
|
per-participant session budget, worst-case sandbox-minutes), `lanes[]` (always present,
|
|
529
529
|
length 1 at N=1; per-participant status/session/sandbox/subject), and `laneSummary`
|
|
@@ -668,7 +668,7 @@ Adapter-owned fields:
|
|
|
668
668
|
Abbreviated synthetic fixture (omits `cwd`, `simulations`, `streams` and `events`):
|
|
669
669
|
|
|
670
670
|
```yaml
|
|
671
|
-
schema: humanish.run-bundle.
|
|
671
|
+
schema: humanish.run-bundle.v4
|
|
672
672
|
runId: synthetic-run-bundle-2026-06-02t10-00-00-000z-proof
|
|
673
673
|
mode: dry-run
|
|
674
674
|
simCount: 1
|
|
@@ -1010,6 +1010,11 @@ text. Older receipts without it remain valid. The phase does not replace the
|
|
|
1010
1010
|
separate dispatch, usage, or cleanup evidence, and does not establish the cause
|
|
1011
1011
|
of a timeout. Participant outcome text includes the phase when available.
|
|
1012
1012
|
|
|
1013
|
+
Three fields gain one entry per model request: `tokenUsage.turns`, `conversation.requests` and
|
|
1014
|
+
`providerRequests`. From run bundle v4 they are only in the participant's trace file, and
|
|
1015
|
+
run.json's `streams[].actor` and `streams[].liveActor` leave them out; the trace loader puts them
|
|
1016
|
+
back ([run bundle contract](run-bundle.md#per-request-lists)).
|
|
1017
|
+
|
|
1013
1018
|
Actors execute or simulate the trial. Actor evidence is the provider-neutral
|
|
1014
1019
|
`humanish.actor-trace.v1` (`src/actors/contract.ts`): Codex app-server items,
|
|
1015
1020
|
computer-use cycles, scripted browser steps, and in-sandbox terminal-agent exec
|
|
@@ -1500,8 +1505,9 @@ Three new `.v1` schema tags ship, all additive and optional so
|
|
|
1500
1505
|
the magnitude; the flag propagates into every estimate so a stand-in is never
|
|
1501
1506
|
mistaken for a live rate. An unknown model/desktop rate is declared absent
|
|
1502
1507
|
(`estimatedCostUsd: null` + a `reason`), never guessed.
|
|
1503
|
-
- `humanish.actor-estimated-cost.v1` (`ActorTrace.estimatedCost
|
|
1504
|
-
|
|
1508
|
+
- `humanish.actor-estimated-cost.v1` (`ActorTrace.estimatedCost`, and from run bundle v4
|
|
1509
|
+
`streams[].liveActor.estimatedCost`, which the run writer prices from a running participant's
|
|
1510
|
+
usage rows before leaving them out of run.json): one participant's token-derived model cost, with `estimatedCostUsd` (or `null` + `reason`
|
|
1505
1511
|
`no_rate_for_model`/`no_token_usage`/`account_billing_unknown`), `ratesAsOf`, `source`, `modelId`,
|
|
1506
1512
|
optional `placeholder`, and a `breakdown`. `basis: aggregated_turns_base_rate` marks usage that
|
|
1507
1513
|
arrived as turn totals, such as Codex `turn.completed`, priced at the base tier because no single
|
|
@@ -1585,11 +1591,14 @@ across the study's participants. Either, both, or neither may be declared. Witho
|
|
|
1585
1591
|
shared threshold, N participants each have their own `maxUsd`; N × `maxUsd` is the sum
|
|
1586
1592
|
of those thresholds, not a guaranteed total spending ceiling.
|
|
1587
1593
|
|
|
1588
|
-
The loop checks
|
|
1589
|
-
response's actions
|
|
1590
|
-
|
|
1591
|
-
|
|
1592
|
-
|
|
1594
|
+
The loop checks both thresholds after a model response reports usage, before
|
|
1595
|
+
dispatching that response's actions, and again before each provider request,
|
|
1596
|
+
the first included. The check before a request stops a participant whose
|
|
1597
|
+
sibling crossed `maxTotalUsd` while it was acting or while its desktop was
|
|
1598
|
+
starting. Neither check reserves the next request's worst-case cost. A request
|
|
1599
|
+
already in flight, at most one per running participant, can exceed these
|
|
1600
|
+
estimates. These thresholds are not hard provider billing caps and exclude
|
|
1601
|
+
desktop and target-app charges.
|
|
1593
1602
|
|
|
1594
1603
|
A capped session sends a further request only while every earlier request's
|
|
1595
1604
|
usage is known or bounded:
|
|
@@ -1617,8 +1626,9 @@ only; `tokenUsage` and the cost summary keep the reported usage.
|
|
|
1617
1626
|
Crossing `maxUsd` ends `budget_reached` / `incomplete`, including when no
|
|
1618
1627
|
material action has executed. The recorded reason distinguishes prior progress
|
|
1619
1628
|
from no material progress; a harness budget stop is not participant abandonment.
|
|
1620
|
-
Crossing the shared study threshold ends `budget_reached` / `incomplete
|
|
1621
|
-
sibling
|
|
1629
|
+
Crossing the shared study threshold ends `budget_reached` / `incomplete`. Each
|
|
1630
|
+
sibling participant stops before it sends its next request; one with a request
|
|
1631
|
+
in flight stops when that reply arrives, before its actions run. Reaching a
|
|
1622
1632
|
threshold is not proof of task completion.
|
|
1623
1633
|
|
|
1624
1634
|
An absent threshold is uncapped. A zero threshold can still permit a paid model
|
|
@@ -1628,7 +1638,7 @@ mode. Use the keyless `humanish run first-run` preview or an explicit
|
|
|
1628
1638
|
threshold on a model `src/run/pricing.ts` cannot price is refused at preflight
|
|
1629
1639
|
(`HUMANISH_COMPUTER_USE_UNPRICED_CAP`, or `HUMANISH_SHARED_WORLD_UNPRICED_CAP` on a
|
|
1630
1640
|
shared-world study) before sandbox allocation. This rate-availability
|
|
1631
|
-
check is separate from the
|
|
1641
|
+
check is separate from the spend checks during the session.
|
|
1632
1642
|
|
|
1633
1643
|
Library callers can make a capped session stricter by composing it from the loop and the default
|
|
1634
1644
|
provider:
|
|
@@ -1907,7 +1917,7 @@ including existence for referenced adapter artifacts.
|
|
|
1907
1917
|
|
|
1908
1918
|
Reserved: `humanish.evidence-stream.v1` has never shipped as a standalone
|
|
1909
1919
|
schema, and streams are not standalone artifacts. They are the `streams` array
|
|
1910
|
-
inside the run bundle (`humanish.run-bundle.
|
|
1920
|
+
inside the run bundle (`humanish.run-bundle.v4`), normalizing UI, browser, terminal, TUI,
|
|
1911
1921
|
code-agent UI, artifact, and summary streams, each with transport, terminal
|
|
1912
1922
|
tail, completion, meaningful-use verdicts, and artifact pointers. See
|
|
1913
1923
|
[`run-bundle.md`](run-bundle.md#completion-and-meaningful-use-verdicts) for
|
|
@@ -367,7 +367,7 @@ and an impression alone supports only a finding about what the participant said.
|
|
|
367
367
|
|
|
368
368
|
## Durable records
|
|
369
369
|
|
|
370
|
-
The `humanish.observer-data.
|
|
370
|
+
The `humanish.observer-data.v3` schema is unchanged by analysis. A companion
|
|
371
371
|
`humanish.study-analysis.v1` artifact records source/config/input hashes,
|
|
372
372
|
participant context, evidence manifest, coverage, provider/model/prompt version,
|
|
373
373
|
status, usage and validated findings. The Observer's projection of it,
|
|
@@ -13,7 +13,7 @@ exactly those cases.
|
|
|
13
13
|
The `observer/` workspace builds with Vite into a single HTML file with fonts inlined and no
|
|
14
14
|
network references. The root build copies it to `dist/observer-app.html`. `renderObserverHtml`
|
|
15
15
|
is the one path every surface (observe, observe --all, watch, studies) uses; it injects the run's
|
|
16
|
-
`humanish.observer-data.
|
|
16
|
+
`humanish.observer-data.v3` snapshot into that file, with the participants' trace items beside it
|
|
17
17
|
gzip-compressed (docs/architecture/observer.md). There is no second renderer and no feature
|
|
18
18
|
flag; rolling back means pinning an older package version.
|
|
19
19
|
|