@letthembuild/run-receipts 0.0.0-stage → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +201 -0
- package/NOTICE +4 -0
- package/PRODUCERS.md +192 -0
- package/README.md +216 -2
- package/SPEC.md +230 -0
- package/examples/agreed-verified.json +26 -0
- package/examples/not-unanimous-failing.json +38 -0
- package/examples/produce.mjs +157 -0
- package/package.json +51 -4
- package/receipt.schema.json +124 -0
- package/sign.mjs +113 -0
- package/verify.mjs +328 -0
package/SPEC.md
ADDED
|
@@ -0,0 +1,230 @@
|
|
|
1
|
+
# Run receipt, version 0.1 (draft)
|
|
2
|
+
|
|
3
|
+
Status: draft. Field names and rules may change before 1.0.
|
|
4
|
+
|
|
5
|
+
## What a receipt is
|
|
6
|
+
|
|
7
|
+
Several models agreeing on an answer is not evidence that the answer is right. A run receipt is the short, machine-readable account of what a multi-model run actually did to check its own work: who took part, which commands were run and how they exited, where each reviewer ended up, and what nobody verified.
|
|
8
|
+
|
|
9
|
+
A receipt holds figures, names and commands. It never holds a prompt, a model's reply, a file's contents or a command's full output.
|
|
10
|
+
|
|
11
|
+
A receipt is the producer's account of the run. A verifier can prove that the account is well-formed, consistent with itself and unchanged since its digest was taken, and, when it is signed, which keys put their name to it. It cannot prove the run happened as described. See "What a verifier cannot tell you".
|
|
12
|
+
|
|
13
|
+
## Vocabulary
|
|
14
|
+
|
|
15
|
+
- **Run**: one request worked on by one lead and zero or more reviewers until it ends.
|
|
16
|
+
- **Seat**: one participant in the run, with a display name and the model behind it.
|
|
17
|
+
- **Lead**: the seat that does the work. **Reviewer**: a seat that examines it.
|
|
18
|
+
- **Check**: a command a seat ran to test the work.
|
|
19
|
+
- **Stance**: where a reviewer ended the run. `agreed`, `objected`, `silent` (answered without saying either way) or `unavailable` (could not answer at all).
|
|
20
|
+
- **Gap**: something that was not verified, stated in words.
|
|
21
|
+
|
|
22
|
+
## The document
|
|
23
|
+
|
|
24
|
+
A receipt is one JSON object, UTF-8. `receipt.schema.json` is the normative shape; unknown fields are not allowed.
|
|
25
|
+
|
|
26
|
+
### Top level
|
|
27
|
+
|
|
28
|
+
| Field | Required | Meaning |
|
|
29
|
+
|---|---|---|
|
|
30
|
+
| `receiptVersion` | yes | `"0.1"`. |
|
|
31
|
+
| `id` | no | Opaque id chosen by the producer. |
|
|
32
|
+
| `producer` | yes | `name` and optional `version` of the tool that wrote the receipt. |
|
|
33
|
+
| `endedAt` | yes | When the run ended, UTC, `YYYY-MM-DDTHH:MM:SSZ`. |
|
|
34
|
+
| `revision` | no | The commit the work ended on, or null outside a repository. May say that uncommitted changes were present. |
|
|
35
|
+
| `durationMs` | yes | Wall-clock length of the run. |
|
|
36
|
+
| `firstReplyMs` | no | Time until the first seat began answering. |
|
|
37
|
+
| `tokens` | yes | Tokens used by all seats, in and out. |
|
|
38
|
+
| `costUsd` | yes | Metered cost of the run in US dollars. Seats on a flat subscription add nothing. |
|
|
39
|
+
| `seats` | yes | One entry per seat, 1 to 32. |
|
|
40
|
+
| `verdict` | yes | How the reviewers ended, or null when nobody reviewed. |
|
|
41
|
+
| `checks` | yes | The commands run to test the work. May be empty. |
|
|
42
|
+
| `totals` | yes | The counts the grade is worked out from. |
|
|
43
|
+
| `grade` | yes | One word for how well the result was checked. |
|
|
44
|
+
| `gaps` | yes | What was not verified. May be empty, and is listed even when everyone agreed. |
|
|
45
|
+
| `digest` | no | `sha256:` and 64 hex characters over the canonical form. |
|
|
46
|
+
| `signatures` | no | One to eight signatures over the digest. See "Signatures". |
|
|
47
|
+
|
|
48
|
+
### Seat
|
|
49
|
+
|
|
50
|
+
| Field | Required | Meaning |
|
|
51
|
+
|---|---|---|
|
|
52
|
+
| `name` | yes | Display name, unique within the run. Checks and holdouts refer to it. |
|
|
53
|
+
| `role` | yes | `lead` or `reviewer`. |
|
|
54
|
+
| `model` | yes | The model identifier the seat ran. |
|
|
55
|
+
| `family` | no | The model's vendor family, lower case. |
|
|
56
|
+
| `turns` | yes | Answers the seat completed. |
|
|
57
|
+
| `errors` | yes | Attempts that failed. |
|
|
58
|
+
| `timeouts` | yes | How many of those errors were the seat going quiet. |
|
|
59
|
+
| `durationMs` | yes | Time the seat spent answering. Seats may run in parallel, so these do not add up to the run's length. |
|
|
60
|
+
| `costUsd` | yes | Metered cost of this seat. |
|
|
61
|
+
| `costBasis` | no | `metered`, `subscription` (a flat plan bought elsewhere) or `unpriced` (no published price). |
|
|
62
|
+
| `stance` | reviewers only | Where the reviewer ended. |
|
|
63
|
+
| `commandsVisible` | no | False when the seat worked inside a tool whose commands the producer could not observe. Such a seat is not counted as having run nothing; it is counted as not visible. |
|
|
64
|
+
|
|
65
|
+
### Verdict
|
|
66
|
+
|
|
67
|
+
| Field | Meaning |
|
|
68
|
+
|---|---|
|
|
69
|
+
| `outcome` | `agreed`, `majority` (more than half agree) or `none` (half or fewer; a tie is not a majority). |
|
|
70
|
+
| `agreed` | How many reviewers agreed. |
|
|
71
|
+
| `reviewers` | How many reviewers the outcome is counted over. |
|
|
72
|
+
| `holdouts` | Every reviewer that did not agree, by name, with its stance. |
|
|
73
|
+
|
|
74
|
+
A reviewer that could not answer is not a reviewer that disagreed. When every reviewer that answered agreed and no check is still failing, the outcome is `agreed`, counted over those that answered, and the missing seats stay named in `holdouts`.
|
|
75
|
+
|
|
76
|
+
### Check
|
|
77
|
+
|
|
78
|
+
| Field | Required | Meaning |
|
|
79
|
+
|---|---|---|
|
|
80
|
+
| `by` | yes | The seat that ran it, or null when unknown. |
|
|
81
|
+
| `alsoBy` | no | Up to four other seats that re-ran it with the same result. |
|
|
82
|
+
| `purpose` | no | What the seat said the command was for. |
|
|
83
|
+
| `command` | yes | The command as run, at most 300 characters. |
|
|
84
|
+
| `exitCode` | yes | Its exit code, or null when none was seen. |
|
|
85
|
+
| `result` | no | One line of its output, at most 200 characters. |
|
|
86
|
+
| `revision` | no | The commit it ran against. |
|
|
87
|
+
| `where` | yes | `local`, `ci`, `cloud` or `other`. A category, never a machine name. |
|
|
88
|
+
| `deployed` | yes | True when the command reached the place the work actually runs. |
|
|
89
|
+
| `missingSetup` | yes | What the command needed and did not have, in words, or null. |
|
|
90
|
+
| `state` | yes | `checked` (ran and answered), `reproduced` (a failure was shown), `blocked` (could not run for want of setup), `unknown` (ran, result not seen) or `untested` (named but never run). |
|
|
91
|
+
| `durationMs` | no | How long it took. |
|
|
92
|
+
|
|
93
|
+
`checks` is a readable list, not a ledger: identical re-runs are merged into one entry with `alsoBy`, and a producer may cap the list. `totals` holds the full counts.
|
|
94
|
+
|
|
95
|
+
### Totals
|
|
96
|
+
|
|
97
|
+
| Field | Required | Meaning |
|
|
98
|
+
|---|---|---|
|
|
99
|
+
| `reviewers` | yes | Reviewers that answered. |
|
|
100
|
+
| `verifiers` | yes | Reviewers that ran at least one command. |
|
|
101
|
+
| `notVisible` | no | Reviewers whose commands could not be observed. |
|
|
102
|
+
| `contradicted` | yes | Reviewers that said they agreed while their own checks failed. Their agreement does not count. |
|
|
103
|
+
| `ran` | yes | Commands run to test the work. |
|
|
104
|
+
| `passed` | yes | Of those, how many passed. |
|
|
105
|
+
| `failed` | yes | How many failed at any point in the run. |
|
|
106
|
+
| `failingNow` | yes | How many were still failing when the run ended. |
|
|
107
|
+
| `expected` | no | Failed as intended: a probe answering "not installed", or the failure the person asked to see. Neither passed nor failing. |
|
|
108
|
+
| `resolved` | no | Failed earlier and passing at the end: caught and fixed. |
|
|
109
|
+
| `filesChanged` | no | Files the lead changed. Absent means not measured; 0 means measured and none. |
|
|
110
|
+
|
|
111
|
+
### Grade
|
|
112
|
+
|
|
113
|
+
One of six words, decided by the first rule that applies:
|
|
114
|
+
|
|
115
|
+
1. `failed`: a check is still failing, or a reviewer contradicted its own checks.
|
|
116
|
+
2. `verified`: commands were run, and every reviewer ran some.
|
|
117
|
+
3. `partial`: commands were run, by some reviewers only.
|
|
118
|
+
4. `reviewed`: nothing was run and no file was changed. A question was answered and read.
|
|
119
|
+
5. `not-visible`: nothing was run where it could be seen, and every reviewer worked out of view.
|
|
120
|
+
6. `unverified`: none of the above.
|
|
121
|
+
|
|
122
|
+
The grade is a word and not a score on purpose. A failed check is a different answer from an unrun one, not fewer points.
|
|
123
|
+
|
|
124
|
+
## What a verifier must check
|
|
125
|
+
|
|
126
|
+
A receipt is valid when it satisfies the schema and every rule below. Rule ids are stable, so tools can report them.
|
|
127
|
+
|
|
128
|
+
**Seats**
|
|
129
|
+
- S1. Seat names are unique.
|
|
130
|
+
- S2. At least one seat is the lead.
|
|
131
|
+
- S3. `timeouts` does not exceed `errors`.
|
|
132
|
+
- S4. Every reviewer has a `stance`; the lead has none.
|
|
133
|
+
- S5. A seat whose `costBasis` is `subscription` or `unpriced` has `costUsd` 0.
|
|
134
|
+
|
|
135
|
+
**Verdict**
|
|
136
|
+
- V1. `verdict` is null exactly when the run has no reviewers.
|
|
137
|
+
- V2. `verdict.agreed` equals the number of reviewers whose stance is `agreed`.
|
|
138
|
+
- V3. `holdouts` is exactly the reviewers whose stance is not `agreed`, each with that stance.
|
|
139
|
+
- V4. `outcome` is `agreed` only if at least one reviewer answered and every reviewer that answered agreed; and, if any reviewer was `unavailable`, only if `totals.failingNow` is 0. Otherwise `outcome` is `majority` when more than half of all reviewers agreed, else `none`.
|
|
140
|
+
- V5. `verdict.reviewers` is the number of reviewers that answered when the outcome is `agreed`, and the number of all reviewers otherwise.
|
|
141
|
+
|
|
142
|
+
A producer may record `majority` or `none` where V4 would allow `agreed`: a run can withhold agreement for reasons of its own. It may never record `agreed` where V4 forbids it.
|
|
143
|
+
|
|
144
|
+
**Totals**
|
|
145
|
+
- T1. `reviewers` equals the number of reviewers whose stance is not `unavailable`.
|
|
146
|
+
- T2. `verifiers`, `notVisible` and `contradicted` do not exceed `reviewers`; `notVisible` does not exceed the reviewers marked `commandsVisible: false`.
|
|
147
|
+
- T3. `passed + failed + expected` does not exceed `ran`.
|
|
148
|
+
- T4. `failingNow` does not exceed `failed`; `resolved` does not exceed `failed - failingNow`.
|
|
149
|
+
- T5. `ran` is above 0 exactly when `verifiers` is above 0.
|
|
150
|
+
|
|
151
|
+
**Grade**
|
|
152
|
+
- G1. `grade` is the word the totals earn under the six rules above.
|
|
153
|
+
|
|
154
|
+
**Checks**
|
|
155
|
+
- C1. `by` and every name in `alsoBy` is a seat in the run.
|
|
156
|
+
- C2. A check that exited 0 has state `checked`.
|
|
157
|
+
- C3. State `blocked` and a non-null `missingSetup` go together.
|
|
158
|
+
- C4. States `unknown` and `untested` have a null `exitCode`.
|
|
159
|
+
- C5 (warning). More checks are listed as run than `totals.ran` counts.
|
|
160
|
+
|
|
161
|
+
**Money and time**
|
|
162
|
+
- M1 (warning). Seat costs differ from `costUsd` by more than one cent.
|
|
163
|
+
- M2. `firstReplyMs` does not exceed `durationMs`.
|
|
164
|
+
- M3. `endedAt` is a real date.
|
|
165
|
+
|
|
166
|
+
**Digest**
|
|
167
|
+
- D1. If `digest` is present it equals the digest of the receipt.
|
|
168
|
+
- D2 (warning). No digest is present.
|
|
169
|
+
|
|
170
|
+
**Signatures**
|
|
171
|
+
- K1. Every signature is its key's signature over this receipt.
|
|
172
|
+
- K2. No key signs twice.
|
|
173
|
+
- K3. Only when the reader requires a signature: at least one signature is from a key the reader trusts.
|
|
174
|
+
|
|
175
|
+
Warnings do not make a receipt invalid.
|
|
176
|
+
|
|
177
|
+
## Digest
|
|
178
|
+
|
|
179
|
+
The digest is SHA-256 over the receipt's canonical form, written `sha256:` followed by lower-case hex. The canonical form is the receipt without its top-level `digest` and `signatures` fields, serialised as JSON with object keys sorted by code unit at every level, no whitespace, and numbers and strings as `JSON.stringify` writes them, encoded as UTF-8.
|
|
180
|
+
|
|
181
|
+
A digest lets a copy of a receipt, such as one quoted in a pull-request comment, be matched against the file it came from. It is not a signature: anyone who edits a receipt can retake its digest.
|
|
182
|
+
|
|
183
|
+
## Signatures
|
|
184
|
+
|
|
185
|
+
A signature lets a reader tell who put their name to a receipt. Signing is optional. Signatures were added to this draft after its first release; a verifier older than 0.2.0 rejects a signed receipt as carrying an unknown field.
|
|
186
|
+
|
|
187
|
+
Each entry in `signatures` has two fields and no others:
|
|
188
|
+
|
|
189
|
+
| Field | Meaning |
|
|
190
|
+
|---|---|
|
|
191
|
+
| `key` | The signer's Ed25519 public key: `ed25519:` followed by its 32 bytes in unpadded base64url. |
|
|
192
|
+
| `sig` | The signature's 64 bytes in unpadded base64url. |
|
|
193
|
+
|
|
194
|
+
What is signed is the UTF-8 bytes of the line `run-receipt-signature-v1`, a line feed, and the receipt's digest as written (`sha256:` and the hex). The digest signed is the one worked out from the receipt, not the one it states, so a signature holds only over the receipt as it stands. The leading line keeps a signature made for a receipt from being valid for anything else the same key signs.
|
|
195
|
+
|
|
196
|
+
Because `signatures` is left out of the canonical form, signing does not change a receipt's digest, and a second party can add its own signature without disturbing the first.
|
|
197
|
+
|
|
198
|
+
### What a signature means
|
|
199
|
+
|
|
200
|
+
A signature that holds says that the holder of that key signed exactly this receipt. That is all it says. Three things follow.
|
|
201
|
+
|
|
202
|
+
- **A key is not a name.** A receipt carries keys and never says whose they are: a signature entry has no room for a name, so a receipt cannot introduce its own signer. The reader decides which keys to trust and what to call them, in a list the reader keeps. A signature from a key the reader has not listed proves that somebody signed, and nothing about who.
|
|
203
|
+
- **A signature is worth what the key's keeping is worth.** A key held by a build server and never shown to anyone supports a strong claim. A key shipped inside an application that runs on its users' machines can be taken out by any of them, and proves only that the receipt came from a copy of that application. Say which kind your key is when you publish it.
|
|
204
|
+
- **A signature does not make a receipt true.** It moves the question from "was this file edited?" to "do I believe this signer's account?". The signer can still be wrong, or lie.
|
|
205
|
+
|
|
206
|
+
A reader that requires a signature (rule K3) must name the keys it trusts. Requiring a signature from any key at all is meaningless, since anyone can make a key.
|
|
207
|
+
|
|
208
|
+
## What a verifier cannot tell you
|
|
209
|
+
|
|
210
|
+
- That the commands were really run, or exited as stated.
|
|
211
|
+
- That the seats were the models named.
|
|
212
|
+
- That the receipt belongs to the change it is attached to, beyond the `revision` it names.
|
|
213
|
+
|
|
214
|
+
A signature from a key you trust narrows the first two to how far you trust the signer. Nothing in a receipt closes them outright.
|
|
215
|
+
|
|
216
|
+
## What a receipt must not carry
|
|
217
|
+
|
|
218
|
+
A tool that runs models holds far more about a run than a receipt should. A producer must keep all of the following out of a receipt, and the schema rejects unknown fields so that none of it can be added by accident.
|
|
219
|
+
|
|
220
|
+
| Left out | Why |
|
|
221
|
+
|---|---|
|
|
222
|
+
| Prompts, replies, conversation titles, file contents | A receipt is figures, not the work. |
|
|
223
|
+
| Full command output | Output carries paths, tokens and data. One line (`result`) is the limit, and the producer must mask secrets in it and in `command`. |
|
|
224
|
+
| Account email, user id, team or organisation id | Personal and customer data. |
|
|
225
|
+
| The id of the conversation or job the run belonged to, or anything derived from it | Links a public file to a private record. `id` must be opaque. |
|
|
226
|
+
| Machine names and host names for where a check ran | `where` is a four-word category for this reason. |
|
|
227
|
+
| How a seat was paid for beyond `costBasis` | Billing detail. |
|
|
228
|
+
| Network destinations blocked or allowed, browser permissions, sandbox decisions | Describe the person's machine and browsing, not the result. |
|
|
229
|
+
| Team policy and configuration | Belongs to the team, not to the run. |
|
|
230
|
+
| Free-text narrative: why a seat was silent, what failed, notes for later | Free text can quote the work. `gaps` and `purpose` are the only prose, and both are short. |
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
{
|
|
2
|
+
"receiptVersion": "0.1",
|
|
3
|
+
"id": "example-agreed-verified",
|
|
4
|
+
"producer": { "name": "example-producer", "version": "0.0.1" },
|
|
5
|
+
"endedAt": "2026-11-10T16:42:07Z",
|
|
6
|
+
"revision": "4f1c2ab90d3e",
|
|
7
|
+
"durationMs": 184300,
|
|
8
|
+
"firstReplyMs": 2100,
|
|
9
|
+
"tokens": 61240,
|
|
10
|
+
"costUsd": 0.4312,
|
|
11
|
+
"seats": [
|
|
12
|
+
{ "name": "Claude", "role": "lead", "model": "claude-sonnet-5-5", "family": "claude", "turns": 3, "errors": 0, "timeouts": 0, "durationMs": 96200, "costUsd": 0.2871, "costBasis": "metered" },
|
|
13
|
+
{ "name": "Gemini", "role": "reviewer", "model": "gemini-3-pro", "family": "gemini", "turns": 2, "errors": 0, "timeouts": 0, "durationMs": 41800, "costUsd": 0.0935, "costBasis": "metered", "stance": "agreed" },
|
|
14
|
+
{ "name": "ChatGPT", "role": "reviewer", "model": "gpt-5.2", "family": "openai", "turns": 2, "errors": 1, "timeouts": 1, "durationMs": 46300, "costUsd": 0.0506, "costBasis": "metered", "stance": "agreed" }
|
|
15
|
+
],
|
|
16
|
+
"verdict": { "outcome": "agreed", "agreed": 2, "reviewers": 2, "holdouts": [] },
|
|
17
|
+
"checks": [
|
|
18
|
+
{ "by": "Gemini", "alsoBy": ["ChatGPT"], "purpose": "Run the unit tests", "command": "npm test", "exitCode": 0, "result": "42 passing", "revision": "4f1c2ab90d3e", "where": "local", "deployed": false, "missingSetup": null, "state": "checked", "durationMs": 8400 },
|
|
19
|
+
{ "by": "ChatGPT", "purpose": "Check types", "command": "npm run typecheck", "exitCode": 0, "result": "", "revision": "4f1c2ab90d3e", "where": "local", "deployed": false, "missingSetup": null, "state": "checked", "durationMs": 5100 },
|
|
20
|
+
{ "by": "Gemini", "purpose": "Ask the staging health endpoint", "command": "curl -fsS https://staging.example.com/healthz", "exitCode": 0, "result": "ok", "revision": null, "where": "local", "deployed": true, "missingSetup": null, "state": "checked" }
|
|
21
|
+
],
|
|
22
|
+
"totals": { "reviewers": 2, "verifiers": 2, "notVisible": 0, "contradicted": 0, "ran": 5, "passed": 4, "failed": 1, "failingNow": 0, "resolved": 1, "filesChanged": 3 },
|
|
23
|
+
"grade": "verified",
|
|
24
|
+
"gaps": ["Behaviour under production load"],
|
|
25
|
+
"digest": "sha256:12b44c77459b21d7d99682b9a75a620d7ee5aae1ef0f581ce5e0e5df677334ab"
|
|
26
|
+
}
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
{
|
|
2
|
+
"receiptVersion": "0.1",
|
|
3
|
+
"id": "example-not-unanimous-failing",
|
|
4
|
+
"producer": { "name": "example-producer", "version": "0.0.1" },
|
|
5
|
+
"endedAt": "2026-11-10T18:05:51Z",
|
|
6
|
+
"revision": "9a07de41c6b2 + uncommitted changes",
|
|
7
|
+
"durationMs": 402900,
|
|
8
|
+
"tokens": 138700,
|
|
9
|
+
"costUsd": 0.2204,
|
|
10
|
+
"seats": [
|
|
11
|
+
{ "name": "Codex", "role": "lead", "model": "gpt-5.2-codex", "family": "openai", "turns": 4, "errors": 0, "timeouts": 0, "durationMs": 211000, "costUsd": 0, "costBasis": "subscription" },
|
|
12
|
+
{ "name": "Claude", "role": "reviewer", "model": "claude-sonnet-5-5", "family": "claude", "turns": 3, "errors": 0, "timeouts": 0, "durationMs": 98400, "costUsd": 0.2204, "costBasis": "metered", "stance": "objected" },
|
|
13
|
+
{ "name": "Gemini", "role": "reviewer", "model": "gemini-3-pro", "family": "gemini", "turns": 3, "errors": 0, "timeouts": 0, "durationMs": 77100, "costUsd": 0, "costBasis": "subscription", "stance": "agreed", "commandsVisible": false },
|
|
14
|
+
{ "name": "Grok", "role": "reviewer", "model": "grok-5", "family": "xai", "turns": 0, "errors": 2, "timeouts": 2, "durationMs": 0, "costUsd": 0, "costBasis": "unpriced", "stance": "unavailable" }
|
|
15
|
+
],
|
|
16
|
+
"verdict": {
|
|
17
|
+
"outcome": "none",
|
|
18
|
+
"agreed": 1,
|
|
19
|
+
"reviewers": 3,
|
|
20
|
+
"holdouts": [
|
|
21
|
+
{ "name": "Claude", "stance": "objected" },
|
|
22
|
+
{ "name": "Grok", "stance": "unavailable" }
|
|
23
|
+
]
|
|
24
|
+
},
|
|
25
|
+
"checks": [
|
|
26
|
+
{ "by": "Claude", "purpose": "Run the migration tests", "command": "pytest tests/test_migrate.py -q", "exitCode": 1, "result": "1 failed, 11 passed", "revision": "9a07de41c6b2 + uncommitted changes", "where": "local", "deployed": false, "missingSetup": null, "state": "reproduced", "durationMs": 12900 },
|
|
27
|
+
{ "by": "Claude", "purpose": "Run the tests against Postgres", "command": "pytest tests/test_migrate.py -q --db postgres", "exitCode": 1, "result": "connection refused", "revision": "9a07de41c6b2 + uncommitted changes", "where": "local", "deployed": false, "missingSetup": "a service it needs was not reachable", "state": "blocked" },
|
|
28
|
+
{ "by": null, "command": "make lint", "exitCode": null, "result": "", "revision": null, "where": "ci", "deployed": false, "missingSetup": null, "state": "unknown" }
|
|
29
|
+
],
|
|
30
|
+
"totals": { "reviewers": 2, "verifiers": 1, "notVisible": 1, "contradicted": 0, "ran": 3, "passed": 0, "failed": 2, "failingNow": 2, "filesChanged": 6 },
|
|
31
|
+
"grade": "failed",
|
|
32
|
+
"gaps": [
|
|
33
|
+
"Checked locally only; deployed behaviour not verified",
|
|
34
|
+
"Behaviour on the hosted database",
|
|
35
|
+
"Gemini's commands could not be seen"
|
|
36
|
+
],
|
|
37
|
+
"digest": "sha256:1e471ce1e3b03fb7fef58f3d56bd08cee3684c499f002e3ffdd5b64334f5170f"
|
|
38
|
+
}
|
|
@@ -0,0 +1,157 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// Copyright 2026 GPULogy AI LLC
|
|
3
|
+
// SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
//
|
|
5
|
+
// An example producer: the least a tool needs to turn what it saw during a
|
|
6
|
+
// run into a valid receipt. PRODUCERS.md walks through it. It is an example
|
|
7
|
+
// to read and adapt, not a library; the rules it follows are SPEC.md's.
|
|
8
|
+
//
|
|
9
|
+
// node examples/produce.mjs print a receipt for the sample run below
|
|
10
|
+
|
|
11
|
+
import { realpathSync } from "node:fs";
|
|
12
|
+
import { fileURLToPath } from "node:url";
|
|
13
|
+
import { gradeOf, digestOf, verifyReceipt } from "../verify.mjs";
|
|
14
|
+
|
|
15
|
+
/** One line, cut to length: `command` and `result` never carry more. */
|
|
16
|
+
const line = (s, max) => Array.from(String(s ?? "").replace(/\s+/g, " ").trim()).slice(0, max).join("");
|
|
17
|
+
|
|
18
|
+
/**
|
|
19
|
+
* A starting point for masking, and no more than that. A real producer knows
|
|
20
|
+
* what its own secrets look like and must mask those too.
|
|
21
|
+
*/
|
|
22
|
+
export const mask = (s) => String(s ?? "")
|
|
23
|
+
.replace(/\b([A-Z][A-Z0-9_]*(?:KEY|TOKEN|SECRET|PASSWORD)[A-Z0-9_]*)=\S+/g, "$1=***")
|
|
24
|
+
.replace(/\b(Bearer|Basic)\s+[A-Za-z0-9._~+/=-]{8,}/g, "$1 ***")
|
|
25
|
+
.replace(/(?:\/Users|\/home)\/[^/\s]+/g, "~");
|
|
26
|
+
|
|
27
|
+
/**
|
|
28
|
+
* Build a receipt from a run.
|
|
29
|
+
*
|
|
30
|
+
* run.seats every seat, the lead and the reviewers, with its figures and,
|
|
31
|
+
* for a reviewer, where it ended (stance)
|
|
32
|
+
* run.commands every command a reviewer ran to test the work, in the order
|
|
33
|
+
* they were run: { by, command, exitCode, purpose?, result?,
|
|
34
|
+
* where?, deployed?, missingSetup? }
|
|
35
|
+
* run.gaps what nobody verified, in words
|
|
36
|
+
*/
|
|
37
|
+
export function buildReceipt(run) {
|
|
38
|
+
const reviewers = run.seats.filter((s) => s.role === "reviewer");
|
|
39
|
+
const answered = reviewers.filter((s) => s.stance !== "unavailable");
|
|
40
|
+
const agreed = reviewers.filter((s) => s.stance === "agreed");
|
|
41
|
+
const counted = new Set(answered.map((s) => s.name));
|
|
42
|
+
|
|
43
|
+
// Totals count what reviewers that answered ran. Each run of a command is
|
|
44
|
+
// counted; a command run twice is two.
|
|
45
|
+
const runs = run.commands.filter((c) => counted.has(c.by));
|
|
46
|
+
const failedRun = (c) => typeof c.exitCode === "number" && c.exitCode !== 0;
|
|
47
|
+
const last = new Map(); // command -> its last run
|
|
48
|
+
const everFailed = new Set(); // commands that failed at any point
|
|
49
|
+
const lastBySeat = new Map(); // "seat command" -> that seat's last run of it
|
|
50
|
+
for (const c of runs) {
|
|
51
|
+
last.set(c.command, c);
|
|
52
|
+
lastBySeat.set(`${c.by}\n${c.command}`, c);
|
|
53
|
+
if (failedRun(c)) everFailed.add(c.command);
|
|
54
|
+
}
|
|
55
|
+
const failingNow = [...last.values()].filter(failedRun).length;
|
|
56
|
+
const verifiers = answered.filter((s) => runs.some((c) => c.by === s.name));
|
|
57
|
+
|
|
58
|
+
const totals = {
|
|
59
|
+
reviewers: answered.length,
|
|
60
|
+
verifiers: verifiers.length,
|
|
61
|
+
notVisible: answered.filter((s) => s.commandsVisible === false && !verifiers.includes(s)).length,
|
|
62
|
+
// Agreed while a command of its own was still failing: that agreement does not count.
|
|
63
|
+
contradicted: agreed.filter((s) => [...lastBySeat.entries()].some(([k, c]) => k.startsWith(`${s.name}\n`) && failedRun(c))).length,
|
|
64
|
+
ran: runs.length,
|
|
65
|
+
passed: runs.filter((c) => c.exitCode === 0).length,
|
|
66
|
+
failed: runs.filter(failedRun).length,
|
|
67
|
+
failingNow,
|
|
68
|
+
resolved: [...everFailed].filter((cmd) => last.get(cmd).exitCode === 0).length,
|
|
69
|
+
...(Number.isInteger(run.filesChanged) ? { filesChanged: run.filesChanged } : {}),
|
|
70
|
+
};
|
|
71
|
+
|
|
72
|
+
// The verdict follows from the stances (rules V1 to V5).
|
|
73
|
+
let verdict = null;
|
|
74
|
+
if (reviewers.length) {
|
|
75
|
+
const everyAnswerAgreed = answered.length > 0 && answered.every((s) => s.stance === "agreed");
|
|
76
|
+
const isAgreed = everyAnswerAgreed && (answered.length === reviewers.length || failingNow === 0);
|
|
77
|
+
verdict = {
|
|
78
|
+
outcome: isAgreed ? "agreed" : agreed.length * 2 > reviewers.length ? "majority" : "none",
|
|
79
|
+
agreed: agreed.length,
|
|
80
|
+
reviewers: isAgreed ? answered.length : reviewers.length,
|
|
81
|
+
holdouts: reviewers.filter((s) => s.stance !== "agreed").map((s) => ({ name: s.name, stance: s.stance })),
|
|
82
|
+
};
|
|
83
|
+
}
|
|
84
|
+
|
|
85
|
+
// One check per command, as it stood at the end.
|
|
86
|
+
const checks = [...last.values()].map((c) => {
|
|
87
|
+
const others = [...new Set(runs.filter((o) => o.command === c.command && o.by !== c.by && o.exitCode === c.exitCode).map((o) => o.by))].slice(0, 4);
|
|
88
|
+
const blocked = Boolean(c.missingSetup);
|
|
89
|
+
return {
|
|
90
|
+
by: c.by,
|
|
91
|
+
...(others.length ? { alsoBy: others } : {}),
|
|
92
|
+
...(c.purpose ? { purpose: line(c.purpose, 100) } : {}),
|
|
93
|
+
command: line(mask(c.command), 300),
|
|
94
|
+
exitCode: typeof c.exitCode === "number" ? c.exitCode : null,
|
|
95
|
+
result: line(mask(c.result), 200),
|
|
96
|
+
where: c.where ?? "local",
|
|
97
|
+
deployed: c.deployed === true,
|
|
98
|
+
missingSetup: blocked ? line(c.missingSetup, 120) : null,
|
|
99
|
+
state: blocked ? "blocked" : typeof c.exitCode !== "number" ? "unknown" : c.exitCode === 0 ? "checked" : "reproduced",
|
|
100
|
+
};
|
|
101
|
+
});
|
|
102
|
+
|
|
103
|
+
const receipt = {
|
|
104
|
+
receiptVersion: "0.1",
|
|
105
|
+
producer: run.producer,
|
|
106
|
+
endedAt: new Date(run.endedAt).toISOString().replace(/\.\d{3}Z$/, "Z"),
|
|
107
|
+
...(run.revision ? { revision: run.revision } : {}),
|
|
108
|
+
durationMs: run.durationMs,
|
|
109
|
+
tokens: run.tokens,
|
|
110
|
+
costUsd: Number(run.seats.reduce((n, s) => n + s.costUsd, 0).toFixed(4)),
|
|
111
|
+
seats: run.seats,
|
|
112
|
+
verdict,
|
|
113
|
+
checks,
|
|
114
|
+
totals,
|
|
115
|
+
grade: gradeOf(totals),
|
|
116
|
+
gaps: run.gaps.map((g) => line(g, 200)),
|
|
117
|
+
};
|
|
118
|
+
// The digest is taken last, over everything else.
|
|
119
|
+
receipt.digest = digestOf(receipt);
|
|
120
|
+
return receipt;
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
/** A run as a tool might hold it when the run ends. */
|
|
124
|
+
export const sampleRun = {
|
|
125
|
+
producer: { name: "example-producer", version: "0.0.1" },
|
|
126
|
+
endedAt: "2026-11-12T09:30:00Z",
|
|
127
|
+
revision: "4f1c2ab90d3e",
|
|
128
|
+
durationMs: 95000,
|
|
129
|
+
tokens: 41800,
|
|
130
|
+
filesChanged: 2,
|
|
131
|
+
seats: [
|
|
132
|
+
{ name: "Builder", role: "lead", model: "model-a", turns: 2, errors: 0, timeouts: 0, durationMs: 52000, costUsd: 0.21, costBasis: "metered" },
|
|
133
|
+
{ name: "First reviewer", role: "reviewer", model: "model-b", turns: 2, errors: 0, timeouts: 0, durationMs: 30000, costUsd: 0.08, costBasis: "metered", stance: "agreed" },
|
|
134
|
+
{ name: "Second reviewer", role: "reviewer", model: "model-c", turns: 1, errors: 0, timeouts: 0, durationMs: 21000, costUsd: 0, costBasis: "subscription", stance: "agreed" },
|
|
135
|
+
],
|
|
136
|
+
commands: [
|
|
137
|
+
{ by: "First reviewer", command: "npm test", exitCode: 1, purpose: "Run the unit tests", result: "1 failing" },
|
|
138
|
+
{ by: "First reviewer", command: "npm test", exitCode: 0, purpose: "Run the unit tests", result: "18 passing" },
|
|
139
|
+
{ by: "Second reviewer", command: "npm test", exitCode: 0, result: "18 passing" },
|
|
140
|
+
{ by: "Second reviewer", command: "API_TOKEN=abc123 npm run smoke", exitCode: 0, purpose: "Smoke test", result: "ok" },
|
|
141
|
+
],
|
|
142
|
+
gaps: ["Checked locally only; deployed behaviour not verified"],
|
|
143
|
+
};
|
|
144
|
+
|
|
145
|
+
function isMain() {
|
|
146
|
+
if (!process.argv[1]) return false;
|
|
147
|
+
try { return realpathSync(process.argv[1]) === realpathSync(fileURLToPath(import.meta.url)); }
|
|
148
|
+
catch { return false; }
|
|
149
|
+
}
|
|
150
|
+
|
|
151
|
+
if (isMain()) {
|
|
152
|
+
const receipt = buildReceipt(sampleRun);
|
|
153
|
+
// Never write a receipt that does not verify.
|
|
154
|
+
const { valid, errors } = verifyReceipt(receipt);
|
|
155
|
+
if (!valid) { console.error(errors.join("\n")); process.exit(1); }
|
|
156
|
+
console.log(JSON.stringify(receipt, null, 2));
|
|
157
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,53 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@letthembuild/run-receipts",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"
|
|
5
|
-
"
|
|
6
|
-
|
|
3
|
+
"version": "0.2.0",
|
|
4
|
+
"description": "A format for recording how well an AI run checked its own work, and a verifier for it.",
|
|
5
|
+
"license": "Apache-2.0",
|
|
6
|
+
"author": "GPULogy AI LLC",
|
|
7
|
+
"homepage": "https://github.com/gpulogyai/run-receipts#readme",
|
|
8
|
+
"repository": {
|
|
9
|
+
"type": "git",
|
|
10
|
+
"url": "git+https://github.com/gpulogyai/run-receipts.git"
|
|
11
|
+
},
|
|
12
|
+
"bugs": {
|
|
13
|
+
"url": "https://github.com/gpulogyai/run-receipts/issues"
|
|
14
|
+
},
|
|
15
|
+
"type": "module",
|
|
16
|
+
"main": "./verify.mjs",
|
|
17
|
+
"exports": {
|
|
18
|
+
".": "./verify.mjs",
|
|
19
|
+
"./sign": "./sign.mjs",
|
|
20
|
+
"./schema": "./receipt.schema.json"
|
|
21
|
+
},
|
|
22
|
+
"bin": {
|
|
23
|
+
"run-receipts": "verify.mjs",
|
|
24
|
+
"run-receipt-verify": "verify.mjs",
|
|
25
|
+
"run-receipt-sign": "sign.mjs"
|
|
26
|
+
},
|
|
27
|
+
"files": [
|
|
28
|
+
"verify.mjs",
|
|
29
|
+
"sign.mjs",
|
|
30
|
+
"receipt.schema.json",
|
|
31
|
+
"SPEC.md",
|
|
32
|
+
"PRODUCERS.md",
|
|
33
|
+
"examples",
|
|
34
|
+
"NOTICE"
|
|
35
|
+
],
|
|
36
|
+
"engines": {
|
|
37
|
+
"node": ">=18"
|
|
38
|
+
},
|
|
39
|
+
"scripts": {
|
|
40
|
+
"test": "node --test verify.test.mjs sign.test.mjs action.test.mjs produce.test.mjs"
|
|
41
|
+
},
|
|
42
|
+
"keywords": [
|
|
43
|
+
"receipt",
|
|
44
|
+
"verification",
|
|
45
|
+
"ai",
|
|
46
|
+
"agents",
|
|
47
|
+
"code-review",
|
|
48
|
+
"json-schema"
|
|
49
|
+
],
|
|
50
|
+
"publishConfig": {
|
|
51
|
+
"access": "public"
|
|
52
|
+
}
|
|
53
|
+
}
|
|
@@ -0,0 +1,124 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
3
|
+
"$id": "https://letthembuild.com/schemas/run-receipt/0.1/receipt.schema.json",
|
|
4
|
+
"title": "Run receipt",
|
|
5
|
+
"description": "What one multi-model run did and how well its result was checked: who took part, what was run, what passed, and what nobody verified. Figures and commands only; never a prompt, a reply or a file's contents.",
|
|
6
|
+
"type": "object",
|
|
7
|
+
"additionalProperties": false,
|
|
8
|
+
"required": ["receiptVersion", "producer", "endedAt", "durationMs", "tokens", "costUsd", "seats", "verdict", "checks", "totals", "grade", "gaps"],
|
|
9
|
+
"properties": {
|
|
10
|
+
"receiptVersion": { "const": "0.1" },
|
|
11
|
+
"id": { "type": "string", "minLength": 1, "maxLength": 200, "description": "An opaque id chosen by the producer. Carries no meaning to a verifier." },
|
|
12
|
+
"producer": {
|
|
13
|
+
"type": "object",
|
|
14
|
+
"additionalProperties": false,
|
|
15
|
+
"required": ["name"],
|
|
16
|
+
"properties": {
|
|
17
|
+
"name": { "type": "string", "minLength": 1, "maxLength": 60 },
|
|
18
|
+
"version": { "type": "string", "minLength": 1, "maxLength": 40 }
|
|
19
|
+
}
|
|
20
|
+
},
|
|
21
|
+
"endedAt": { "type": "string", "pattern": "^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}(\\.\\d{1,3})?Z$", "description": "When the run ended, UTC, RFC 3339." },
|
|
22
|
+
"revision": { "type": ["string", "null"], "maxLength": 60, "description": "The commit the work ended on, if it was in a repository." },
|
|
23
|
+
"durationMs": { "type": "integer", "minimum": 0, "maximum": 86400000 },
|
|
24
|
+
"firstReplyMs": { "type": "integer", "minimum": 0, "maximum": 86400000 },
|
|
25
|
+
"tokens": { "type": "integer", "minimum": 0, "maximum": 2000000000 },
|
|
26
|
+
"costUsd": { "type": "number", "minimum": 0, "maximum": 1000000 },
|
|
27
|
+
"seats": { "type": "array", "minItems": 1, "maxItems": 32, "items": { "$ref": "#/$defs/seat" } },
|
|
28
|
+
"verdict": { "anyOf": [{ "type": "null" }, { "$ref": "#/$defs/verdict" }] },
|
|
29
|
+
"checks": { "type": "array", "maxItems": 200, "items": { "$ref": "#/$defs/check" } },
|
|
30
|
+
"totals": { "$ref": "#/$defs/totals" },
|
|
31
|
+
"grade": { "enum": ["failed", "verified", "partial", "not-visible", "reviewed", "unverified"] },
|
|
32
|
+
"gaps": { "type": "array", "maxItems": 50, "items": { "type": "string", "minLength": 1, "maxLength": 200 } },
|
|
33
|
+
"digest": { "type": "string", "pattern": "^sha256:[0-9a-f]{64}$" },
|
|
34
|
+
"signatures": { "type": "array", "minItems": 1, "maxItems": 8, "items": { "$ref": "#/$defs/signature" }, "description": "Signatures over the digest. Left out of the digest themselves." }
|
|
35
|
+
},
|
|
36
|
+
"$defs": {
|
|
37
|
+
"signature": {
|
|
38
|
+
"type": "object",
|
|
39
|
+
"additionalProperties": false,
|
|
40
|
+
"required": ["key", "sig"],
|
|
41
|
+
"properties": {
|
|
42
|
+
"key": { "type": "string", "pattern": "^ed25519:[A-Za-z0-9_-]{43}$", "description": "The signer's Ed25519 public key: its 32 bytes in base64url." },
|
|
43
|
+
"sig": { "type": "string", "pattern": "^[A-Za-z0-9_-]{86}$", "description": "The signature's 64 bytes in base64url." }
|
|
44
|
+
}
|
|
45
|
+
},
|
|
46
|
+
"stance": { "enum": ["agreed", "objected", "silent", "unavailable"] },
|
|
47
|
+
"seat": {
|
|
48
|
+
"type": "object",
|
|
49
|
+
"additionalProperties": false,
|
|
50
|
+
"required": ["name", "role", "model", "turns", "errors", "timeouts", "durationMs", "costUsd"],
|
|
51
|
+
"properties": {
|
|
52
|
+
"name": { "type": "string", "minLength": 1, "maxLength": 60 },
|
|
53
|
+
"role": { "enum": ["lead", "reviewer"] },
|
|
54
|
+
"model": { "type": "string", "minLength": 1, "maxLength": 120 },
|
|
55
|
+
"family": { "type": "string", "minLength": 1, "maxLength": 40 },
|
|
56
|
+
"turns": { "type": "integer", "minimum": 0, "maximum": 1000 },
|
|
57
|
+
"errors": { "type": "integer", "minimum": 0, "maximum": 1000 },
|
|
58
|
+
"timeouts": { "type": "integer", "minimum": 0, "maximum": 1000 },
|
|
59
|
+
"durationMs": { "type": "integer", "minimum": 0, "maximum": 86400000 },
|
|
60
|
+
"costUsd": { "type": "number", "minimum": 0, "maximum": 1000000 },
|
|
61
|
+
"costBasis": { "enum": ["metered", "subscription", "unpriced"] },
|
|
62
|
+
"stance": { "$ref": "#/$defs/stance" },
|
|
63
|
+
"commandsVisible": { "type": "boolean", "description": "False when the seat worked inside a tool whose commands the producer could not see." }
|
|
64
|
+
}
|
|
65
|
+
},
|
|
66
|
+
"holdout": {
|
|
67
|
+
"type": "object",
|
|
68
|
+
"additionalProperties": false,
|
|
69
|
+
"required": ["name", "stance"],
|
|
70
|
+
"properties": {
|
|
71
|
+
"name": { "type": "string", "minLength": 1, "maxLength": 60 },
|
|
72
|
+
"stance": { "enum": ["objected", "silent", "unavailable"] }
|
|
73
|
+
}
|
|
74
|
+
},
|
|
75
|
+
"verdict": {
|
|
76
|
+
"type": "object",
|
|
77
|
+
"additionalProperties": false,
|
|
78
|
+
"required": ["outcome", "agreed", "reviewers", "holdouts"],
|
|
79
|
+
"properties": {
|
|
80
|
+
"outcome": { "enum": ["agreed", "majority", "none"] },
|
|
81
|
+
"agreed": { "type": "integer", "minimum": 0, "maximum": 32 },
|
|
82
|
+
"reviewers": { "type": "integer", "minimum": 0, "maximum": 32 },
|
|
83
|
+
"holdouts": { "type": "array", "maxItems": 32, "items": { "$ref": "#/$defs/holdout" } }
|
|
84
|
+
}
|
|
85
|
+
},
|
|
86
|
+
"check": {
|
|
87
|
+
"type": "object",
|
|
88
|
+
"additionalProperties": false,
|
|
89
|
+
"required": ["by", "command", "exitCode", "where", "deployed", "missingSetup", "state"],
|
|
90
|
+
"properties": {
|
|
91
|
+
"by": { "type": ["string", "null"], "maxLength": 60 },
|
|
92
|
+
"alsoBy": { "type": "array", "maxItems": 4, "items": { "type": "string", "minLength": 1, "maxLength": 60 } },
|
|
93
|
+
"purpose": { "type": "string", "minLength": 1, "maxLength": 100 },
|
|
94
|
+
"command": { "type": "string", "minLength": 1, "maxLength": 300 },
|
|
95
|
+
"exitCode": { "type": ["integer", "null"] },
|
|
96
|
+
"result": { "type": "string", "maxLength": 200, "description": "One line of the command's output. Never the whole output." },
|
|
97
|
+
"revision": { "type": ["string", "null"], "maxLength": 60 },
|
|
98
|
+
"where": { "enum": ["local", "ci", "cloud", "other"] },
|
|
99
|
+
"deployed": { "type": "boolean", "description": "True when the command reached the place the work actually runs." },
|
|
100
|
+
"missingSetup": { "type": ["string", "null"], "maxLength": 120 },
|
|
101
|
+
"state": { "enum": ["checked", "reproduced", "blocked", "unknown", "untested"] },
|
|
102
|
+
"durationMs": { "type": "integer", "minimum": 0, "maximum": 86400000 }
|
|
103
|
+
}
|
|
104
|
+
},
|
|
105
|
+
"totals": {
|
|
106
|
+
"type": "object",
|
|
107
|
+
"additionalProperties": false,
|
|
108
|
+
"required": ["reviewers", "verifiers", "ran", "passed", "failed", "failingNow", "contradicted"],
|
|
109
|
+
"properties": {
|
|
110
|
+
"reviewers": { "type": "integer", "minimum": 0, "maximum": 32 },
|
|
111
|
+
"verifiers": { "type": "integer", "minimum": 0, "maximum": 32 },
|
|
112
|
+
"notVisible": { "type": "integer", "minimum": 0, "maximum": 32 },
|
|
113
|
+
"contradicted": { "type": "integer", "minimum": 0, "maximum": 32 },
|
|
114
|
+
"ran": { "type": "integer", "minimum": 0, "maximum": 100000 },
|
|
115
|
+
"passed": { "type": "integer", "minimum": 0, "maximum": 100000 },
|
|
116
|
+
"failed": { "type": "integer", "minimum": 0, "maximum": 100000 },
|
|
117
|
+
"failingNow": { "type": "integer", "minimum": 0, "maximum": 100000 },
|
|
118
|
+
"expected": { "type": "integer", "minimum": 0, "maximum": 100000 },
|
|
119
|
+
"resolved": { "type": "integer", "minimum": 0, "maximum": 100000 },
|
|
120
|
+
"filesChanged": { "type": "integer", "minimum": 0, "maximum": 100000 }
|
|
121
|
+
}
|
|
122
|
+
}
|
|
123
|
+
}
|
|
124
|
+
}
|