brainchain-verify 0.3.2 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +30 -13
- package/dist/cli.js +44 -22
- package/dist/verify.d.ts +7 -0
- package/dist/verify.js +263 -54
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -22,8 +22,8 @@ brainchain-verify <mainnet|testnet|chainId> --task <taskId> [--set <TaskBoard>]
|
|
|
22
22
|
| `mainnet`, `testnet` or a chain id | `mainnet` is chain 274663 at `https://rpc.usebrainchain.com`; `testnet` is chain 274630 at `https://rpc.testnet.usebrainchain.com`. |
|
|
23
23
|
| `<decisionId>` | The decision's number in its `DecisionLog`, from 1. |
|
|
24
24
|
| `--task <id>` | Check every decision about one task, oldest first. |
|
|
25
|
-
| `--set <TaskBoard>` | A retired contract set, by its TaskBoard address, by the key a decision page uses (`?set=v3-1`) or by release (`3.1.0`). Without it, the active set. Each set numbers its tasks and decisions from 1. |
|
|
26
|
-
| `--rpc <url>` | Another RPC for the same chain. |
|
|
25
|
+
| `--set <TaskBoard>` | A retired contract set from 3.0.0 on, by its TaskBoard address, by the key a decision page uses (`?set=v3-1`) or by release (`3.1.0`). Without it, the active set. Each set numbers its tasks and decisions from 1. The `v2` set is not supported: its board named tasks by id alone, which the subject check does not read. |
|
|
26
|
+
| `--rpc <url>` | Another RPC for the same chain. Records and briefs on private addresses are read only when this RPC is on one too. |
|
|
27
27
|
| `--json` | One JSON object on stdout. |
|
|
28
28
|
|
|
29
29
|
The package carries the active and retired contract sets from [`deployments/`](../../deployments) as they stood when it was built. An unknown `--set` lists them.
|
|
@@ -46,42 +46,58 @@ Decision 22 on BrainChain mainnet, contract set v3-1 (DecisionLog 0xbe8E68EbDEaf
|
|
|
46
46
|
ok the question quotes task 11's brief, as committed on chain
|
|
47
47
|
ok the levels Jev scored against are the brief's, lowest first
|
|
48
48
|
ok confidence recomputed: paid levels 1, 2 hold 0.46 + 0.34 = 0.8000, so accept at 0.8000
|
|
49
|
-
ok threshold: accept at 80% is at or above
|
|
49
|
+
ok threshold: accept at 80% is at or above task 11's 70% grade threshold, so the work is accepted, as the board recorded
|
|
50
50
|
|
|
51
51
|
Verdict: match. The published record is exactly what the chain recorded, and the chain applied it as its contracts require.
|
|
52
52
|
Not checked: that Jev produced the answer. See https://docs.usebrainchain.com/trust-model
|
|
53
53
|
```
|
|
54
54
|
|
|
55
|
-
Each line is one check. `ok` passed and `FAIL` means the record and the chain disagree. `skip` is a check that could not run, with the reason on the line. `ERROR` means the record could not be fetched, so the checks that need it did not run: the host did not answer
|
|
55
|
+
Each line is one check. `ok` passed and `FAIL` means the record and the chain disagree. `skip` is a check that could not run, with the reason on the line. `ERROR` means the record could not be fetched, so the checks that need it did not run: the host did not answer in up to three tries and 30 seconds in all, it no longer has the record, or the tool refused to read it (see `fetch`).
|
|
56
|
+
|
|
57
|
+
The verdict is `match`, `withheld`, `UNVERIFIED` or `MISMATCH`. `withheld` means a record served without the work passed the checks that could run, and says what they leave out.
|
|
56
58
|
|
|
57
59
|
## What it checks
|
|
58
60
|
|
|
59
61
|
| Check | What passes |
|
|
60
62
|
| --- | --- |
|
|
61
63
|
| `read` | `DecisionLog.get(id)` and the decision's `Recorded` event agree. |
|
|
62
|
-
| `fetch` | The record at the event's `payloadURI` (https, http or a base64 `data:` URI, with redirects refused) is a decision record. |
|
|
64
|
+
| `fetch` | The record at the event's `payloadURI` (https, http or a base64 `data:` URI, with redirects refused) is a decision record of at most 4 MB. Briefs are read the same way, up to 256 KB. Anything on a private or loopback address is refused, however its name resolves, unless `--rpc` is on one too. |
|
|
65
|
+
| `withheld` | On a record served without the work: the kind is a grade or ruling, the request carries the service's marker where the work was, and the chain shows the work is not public. If any of these fails, the record is held to its hashes like any other. |
|
|
63
66
|
| `record-hash` | `keccak256` of the fetched bytes equals `payloadHash`. |
|
|
64
67
|
| `input-hash` | `keccak256` of the record's `request`, serialized canonically (keys sorted, no whitespace), equals `inputHash`. |
|
|
65
68
|
| `kind` | The record's kind is the one on chain. |
|
|
66
|
-
| `subject` | A task decision names `TaskBoard.subjectOf(task)` for this chain and board. A change names its exact call. |
|
|
69
|
+
| `subject` | A task decision names `TaskBoard.subjectOf(task)` for this chain and board. A change names its exact call. A grade of outside work is matched by the rules below. |
|
|
67
70
|
| `options` | The probabilities cover exactly the options in the question. |
|
|
68
71
|
| `answer` | The record's answer encodes to the one on chain: the hired model's id or worker address, or 1 for accept, pay, adopt and allow and 0 for their opposites. |
|
|
69
72
|
| `confidence` | The record's confidence times 10,000, rounded, equals `confidenceBps`. |
|
|
70
73
|
| `top-option` | For a hire, ruling or change, the answer is Jev's most likely option. |
|
|
71
74
|
| `confidence-recomputed` | The confidence follows from the probabilities. A hire uses `(p - 1/n) / (1 - 1/n)`, within 0.02 because the probabilities are rounded to two decimals. A grade sums the probabilities on the paid levels. A change uses the chosen option's probability, and so does a ruling recorded before rulings were graded twice. A grade or ruling with a `second` or `checks` field follows the two-grader rule: the probability of paying is the lower of Jev's and the second grader's, zero when the brief's checks failed, and the answer and confidence follow it. |
|
|
72
75
|
| `second-grader` | On a decision graded twice: the second grader's probability of paying recomputes from its probabilities, and the receipt id it carries is shown. |
|
|
73
|
-
| `brief` | The question quotes the task's brief, fetched from its `Posted` event and checked against `briefHash`. |
|
|
74
|
-
| `levels` | A grade or ruling used the brief's own quality levels and pass level. |
|
|
75
|
-
| `threshold` | The
|
|
76
|
+
| `brief` | The question quotes the task's brief, fetched from its `Posted` event and checked against `briefHash`. Bytes that miss the hash are a `FAIL`; a host that cannot be read is a `skip`. |
|
|
77
|
+
| `levels` | A grade or ruling used the brief's own quality levels and pass level. On a grade of outside work, the pass level is a whole number naming one of the question's levels above the lowest. |
|
|
78
|
+
| `threshold` | The task's threshold, applied as `TaskBoard.hire`, `grade`, `rule` and `ruleAppeal` do, gives the outcome the board emitted. A poster can raise the grade and ruling thresholds while the task is open, so they can be higher than when it was posted; they cannot change after the hire. A ruling on appeal (set 3.5.0 and later) is checked against the board's `AppealRuled` event. A change is queued only on a yes. |
|
|
76
79
|
| `change-subject`, `change-call` | A queued change is the call the decision names and the call Jev was shown. |
|
|
77
80
|
|
|
78
81
|
Some checks skip by design:
|
|
79
82
|
|
|
80
|
-
- A record for work Jev did not accept is served without the work, so `record-hash` and `input-hash` cannot match it. The answer, confidence and threshold are still checked.
|
|
81
83
|
- A decision the board never applied, because a later decision or a closed window settled the task, skips `threshold`.
|
|
82
|
-
- A grade of work delivered outside BrainChain has no task on the board, so it skips `threshold`. Its subject, levels and confidence are still checked.
|
|
83
84
|
- A grade recorded before the chain gated on the paid levels skips `confidence-recomputed`: its confidence is Jev's score confidence, which the probabilities alone do not give.
|
|
84
85
|
|
|
86
|
+
### Records served without the work
|
|
87
|
+
|
|
88
|
+
Work stays private until it is accepted, so a grade or ruling on work that was not accepted is served with the work taken out and the header `x-brainchain-withheld: result`. That copy cannot match `payloadHash` or `inputHash`, so `record-hash` and `input-hash` skip and the verdict is `withheld`, never `match`. Exit status stays 0.
|
|
89
|
+
|
|
90
|
+
The header alone proves nothing, so the tool honours it only under the `withheld` check above. The record's answer and confidence are still held to the chain, and for a task so are its brief, levels and threshold. Nothing ties the rest of the copy to the chain: the question, the split of the probabilities behind the confidence, the second grader. On those you are relying on the host. The chain's rules are the records service's own: a task's work is public once it is paid or graded accepted (a house task's once graded), a paid grade's once accepted, and the delivery in an ACP evaluation never.
|
|
91
|
+
|
|
92
|
+
### Grades of outside work
|
|
93
|
+
|
|
94
|
+
A grade of work delivered outside BrainChain has no task on the board. The record's `subject` is a number, and it is checked against the decision's subject on chain by one of two rules:
|
|
95
|
+
|
|
96
|
+
- A paid grade: the number is the grade's id, and the subject is `keccak256(abi.encode("x402-grade", id))`, with the id as `bytes32`.
|
|
97
|
+
- An ACP evaluation: the number is the job, and the subject is `keccak256(abi.encode("acp-evaluation", chainId, job))`, for job chains 8453, 84532, 4663 and 46630.
|
|
98
|
+
|
|
99
|
+
Its answer and confidence are checked, and the confidence is recomputed from the record's own levels and pass level. Nothing on chain says what those levels should be, so `levels` skips and `threshold` skips: `payloadHash` commits them, but no brief does. The caller who paid for the grade received its verdict directly.
|
|
100
|
+
|
|
85
101
|
## What it cannot check
|
|
86
102
|
|
|
87
103
|
That Jev produced the answer. BrainChain records Jev's answers, and the contracts bound what a recorded answer can do. A decision that passes every check has the question, answer and probabilities the chain acted on. The [trust model](https://docs.usebrainchain.com/trust-model) explains what the chain relies on and what bounds it.
|
|
@@ -90,13 +106,13 @@ That Jev produced the answer. BrainChain records Jev's answers, and the contract
|
|
|
90
106
|
|
|
91
107
|
| Code | Meaning |
|
|
92
108
|
| --- | --- |
|
|
93
|
-
| 0 | Every check that could run passed. Skips do not fail a decision. |
|
|
109
|
+
| 0 | Every check that could run passed. Skips do not fail a decision. A `withheld` verdict also exits 0, so read the verdict if you need to tell it from a full match. |
|
|
94
110
|
| 1 | A check failed, a record could not be fetched, or the decision could not be read: an unknown id, a task with no decisions, an RPC error. |
|
|
95
111
|
| 2 | Bad arguments, or an RPC that serves another chain. |
|
|
96
112
|
|
|
97
113
|
## JSON
|
|
98
114
|
|
|
99
|
-
`--json` prints one object. `decisions` holds one entry, or one per decision with `--task`. `verdict` is `match`, `mismatch` or `unverified` (a record could not be fetched). Each check keeps its `name` across releases, and its `status` is `ok`, `fail`, `skip` or `error`. When the tool cannot run, it prints `{ "error": "..." }` and exits 1. Trimmed to one check:
|
|
115
|
+
`--json` prints one object. `decisions` holds one entry, or one per decision with `--task`. `verdict` is `match`, `withheld`, `mismatch` or `unverified` (a record could not be fetched). With `withheld`, `withheldNotChecked` says what was left out, and the decision carries `withheld: true`. A grade of outside work carries `outside: true`. Each check keeps its `name` across releases, and its `status` is `ok`, `fail`, `skip` or `error`. When the tool cannot run, it prints `{ "error": "..." }` and exits 1. Trimmed to one check:
|
|
100
116
|
|
|
101
117
|
```json
|
|
102
118
|
{
|
|
@@ -139,6 +155,7 @@ That Jev produced the answer. BrainChain records Jev's answers, and the contract
|
|
|
139
155
|
```sh
|
|
140
156
|
npm install
|
|
141
157
|
npm run build
|
|
158
|
+
npm test
|
|
142
159
|
```
|
|
143
160
|
|
|
144
161
|
`build` reads `deployments/*.json` at the repository root into `src/deployments.ts` and compiles to `dist/`. `npm run abi` regenerates `src/abi.ts` from a `forge build` at the root.
|
package/dist/cli.js
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
import { createPublicClient, http, isAddress } from "viem";
|
|
3
3
|
import { decisionLogAbi, taskBoardAbi } from "./abi.js";
|
|
4
4
|
import { NETWORKS } from "./deployments.js";
|
|
5
|
-
import { decisionsOf, errorMessage, verifyDecision } from "./verify.js";
|
|
5
|
+
import { clean, decisionsOf, errorMessage, outcomeOf, verifyDecision, WITHHELD_NOT_CHECKED } from "./verify.js";
|
|
6
6
|
const USAGE = `Usage:
|
|
7
7
|
brainchain-verify <mainnet|testnet|chainId> <decisionId> [--set <TaskBoard>] [--rpc <url>] [--json]
|
|
8
8
|
brainchain-verify <mainnet|testnet|chainId> --task <taskId> [--set <TaskBoard>] [--rpc <url>] [--json]
|
|
@@ -12,14 +12,18 @@ fetches the published record, recomputes its hashes and confidence, and applies
|
|
|
12
12
|
the task's thresholds as the contracts do. Needs no keys.
|
|
13
13
|
|
|
14
14
|
--task <id> check every decision about one task
|
|
15
|
-
--set <board> a retired contract set, by TaskBoard address, by
|
|
16
|
-
the site (v3-1) or by release (3.1.0); without it, the
|
|
17
|
-
|
|
15
|
+
--set <board> a retired contract set from 3.0.0 on, by TaskBoard address, by
|
|
16
|
+
its key on the site (v3-1) or by release (3.1.0); without it, the
|
|
17
|
+
active set
|
|
18
|
+
--rpc <url> another RPC for the same chain; records and briefs on private
|
|
19
|
+
addresses are read only when this is on one too
|
|
18
20
|
--json one JSON object on stdout
|
|
19
21
|
|
|
20
22
|
Exit status: 0 when every check that could run matches, 1 on a mismatch or when
|
|
21
23
|
a decision or its record cannot be read, 2 on bad arguments.`;
|
|
22
24
|
const TRUST_MODEL = "https://docs.usebrainchain.com/trust-model";
|
|
25
|
+
// Sets before 3.0.0 named tasks by id alone, which the subject check does not model, so the package carries none.
|
|
26
|
+
const BEFORE_3 = /^(v2(-|$)|2\.|0x83b31d99fa31a9e5257955a737ef34c40f1c57bf$)/i;
|
|
23
27
|
function usage(message) {
|
|
24
28
|
console.error(`${message}\n\n${USAGE}`);
|
|
25
29
|
process.exit(2);
|
|
@@ -45,9 +49,12 @@ if (json)
|
|
|
45
49
|
const taskArg = flag("--task");
|
|
46
50
|
const setArg = flag("--set");
|
|
47
51
|
const rpcArg = flag("--rpc");
|
|
48
|
-
|
|
52
|
+
// -1 is a bad id, not an option; the id check below says so.
|
|
53
|
+
const unknown = args.find((arg) => arg.startsWith("-") && !/^-\d/.test(arg));
|
|
49
54
|
if (unknown)
|
|
50
55
|
usage(`Unknown option ${unknown}`);
|
|
56
|
+
if (rpcArg !== undefined && !(URL.canParse(rpcArg) && ["http:", "https:"].includes(new URL(rpcArg).protocol)))
|
|
57
|
+
usage("--rpc needs an http or https URL");
|
|
51
58
|
const [networkArg, idArg, ...extra] = args;
|
|
52
59
|
if (networkArg === undefined)
|
|
53
60
|
usage("Name a network: mainnet, testnet or a chain id");
|
|
@@ -58,7 +65,9 @@ if (target === undefined)
|
|
|
58
65
|
usage("Name a decision id, or a task with --task");
|
|
59
66
|
if (!/^[1-9]\d*$/.test(target))
|
|
60
67
|
usage(`${taskArg === undefined ? "A decision" : "A task"} id is a whole number from 1`);
|
|
61
|
-
const foundNetwork =
|
|
68
|
+
const foundNetwork = Object.hasOwn(NETWORKS, networkArg)
|
|
69
|
+
? NETWORKS[networkArg]
|
|
70
|
+
: Object.values(NETWORKS).find((n) => String(n.chainId) === networkArg);
|
|
62
71
|
if (!foundNetwork)
|
|
63
72
|
usage(`Unknown network "${networkArg}": use mainnet, testnet, 274663 or 274630`);
|
|
64
73
|
const network = foundNetwork;
|
|
@@ -66,6 +75,8 @@ const foundSet = setArg === undefined
|
|
|
66
75
|
? network.sets[0]
|
|
67
76
|
: network.sets.find((s) => (isAddress(setArg) ? s.board.toLowerCase() === setArg.toLowerCase() : s.key === setArg || s.release === setArg));
|
|
68
77
|
if (!foundSet) {
|
|
78
|
+
if (setArg !== undefined && BEFORE_3.test(setArg))
|
|
79
|
+
usage(`Contract set ${setArg} predates release 3.0.0, which this tool does not read: those boards named tasks by id alone.`);
|
|
69
80
|
const sets = network.sets.map((s, i) => `${s.key} (TaskBoard ${s.board}${i === 0 ? ", active" : ""})`);
|
|
70
81
|
usage(`No contract set ${setArg} on ${network.name}. Sets: ${sets.join(", ")}`);
|
|
71
82
|
}
|
|
@@ -78,40 +89,50 @@ const ctx = {
|
|
|
78
89
|
};
|
|
79
90
|
function fail(message) {
|
|
80
91
|
if (json)
|
|
81
|
-
console.log(JSON.stringify({ error: message }, null, 2));
|
|
92
|
+
console.log(JSON.stringify({ error: clean(message) }, null, 2));
|
|
82
93
|
else
|
|
83
|
-
console.error(message);
|
|
94
|
+
console.error(clean(message));
|
|
84
95
|
process.exit(1);
|
|
85
96
|
}
|
|
86
97
|
const TAGS = { ok: "ok", fail: "FAIL", skip: "skip", error: "ERROR" };
|
|
87
98
|
const tally = (checks, status) => checks.filter((c) => c.status === status).length;
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
+
function verdict(reports) {
|
|
100
|
+
const all = reports.flatMap((r) => r.checks);
|
|
101
|
+
const failed = tally(all, "fail");
|
|
102
|
+
const many = reports.length > 1;
|
|
103
|
+
switch (outcomeOf(reports)) {
|
|
104
|
+
case "mismatch":
|
|
105
|
+
return `Verdict: MISMATCH. ${failed} check${failed === 1 ? "" : "s"} failed; see the FAIL lines.`;
|
|
106
|
+
case "unverified":
|
|
107
|
+
return "Verdict: UNVERIFIED. A record could not be fetched, so its checks did not run; see the ERROR lines.";
|
|
108
|
+
case "withheld":
|
|
109
|
+
return `Verdict: withheld, not a full match${many ? ` (decision${reports.filter((r) => r.withheld).length === 1 ? "" : "s"} ${reports.filter((r) => r.withheld).map((r) => r.id).join(", ")})` : ""}. ${WITHHELD_NOT_CHECKED}`;
|
|
110
|
+
case "match":
|
|
111
|
+
if (tally(all, "skip") > 0)
|
|
112
|
+
return "Verdict: match. Every check that could run passed; the skip lines say what could not be checked.";
|
|
113
|
+
return many
|
|
114
|
+
? "Verdict: match. The published records are exactly what the chain recorded, and the chain applied them as its contracts require."
|
|
115
|
+
: "Verdict: match. The published record is exactly what the chain recorded, and the chain applied it as its contracts require.";
|
|
116
|
+
}
|
|
99
117
|
}
|
|
100
118
|
function report(reports, header, task) {
|
|
101
119
|
const all = reports.flatMap((r) => r.checks);
|
|
102
120
|
const failed = tally(all, "fail");
|
|
103
121
|
const errors = tally(all, "error");
|
|
104
122
|
const skipped = tally(all, "skip");
|
|
123
|
+
const overall = outcomeOf(reports);
|
|
105
124
|
if (json) {
|
|
106
125
|
const decisions = reports.map(({ checks, ...r }) => ({
|
|
107
126
|
id: r.id,
|
|
108
127
|
kind: r.kind,
|
|
109
128
|
task: r.task,
|
|
129
|
+
...(r.outside ? { outside: true } : {}),
|
|
130
|
+
...(r.withheld ? { withheld: true } : {}),
|
|
110
131
|
block: r.block,
|
|
111
132
|
recordedAt: r.recordedAt,
|
|
112
133
|
confidenceBps: r.confidenceBps,
|
|
113
134
|
payloadURI: r.payloadURI,
|
|
114
|
-
verdict:
|
|
135
|
+
verdict: outcomeOf([{ ...r, checks }]),
|
|
115
136
|
checks,
|
|
116
137
|
}));
|
|
117
138
|
const out = {
|
|
@@ -119,11 +140,12 @@ function report(reports, header, task) {
|
|
|
119
140
|
chainId: network.chainId,
|
|
120
141
|
set,
|
|
121
142
|
...(task === undefined ? {} : { task }),
|
|
122
|
-
verdict:
|
|
143
|
+
verdict: overall,
|
|
123
144
|
failed,
|
|
124
145
|
errors,
|
|
125
146
|
skipped,
|
|
126
147
|
decisions,
|
|
148
|
+
...(overall === "withheld" ? { withheldNotChecked: WITHHELD_NOT_CHECKED } : {}),
|
|
127
149
|
notChecked: "That Jev produced the answer. BrainChain records Jev's answers, and the contracts bound what a recorded answer can do.",
|
|
128
150
|
trustModel: TRUST_MODEL,
|
|
129
151
|
};
|
|
@@ -136,7 +158,7 @@ function report(reports, header, task) {
|
|
|
136
158
|
for (const c of r.checks)
|
|
137
159
|
console.log(` ${TAGS[c.status].padEnd(5)} ${c.message}`);
|
|
138
160
|
}
|
|
139
|
-
console.log(`\n${verdict(
|
|
161
|
+
console.log(`\n${verdict(reports)}`);
|
|
140
162
|
console.log(`Not checked: that Jev produced the answer. See ${TRUST_MODEL}`);
|
|
141
163
|
process.exit(failed + errors > 0 ? 1 : 0);
|
|
142
164
|
}
|
package/dist/verify.d.ts
CHANGED
|
@@ -14,6 +14,8 @@ export interface DecisionReport {
|
|
|
14
14
|
task?: number;
|
|
15
15
|
/** A grade of work delivered outside BrainChain, so no task here. */
|
|
16
16
|
outside?: boolean;
|
|
17
|
+
/** Served without the work, as the chain shows it may be, so its hashes could not be checked. */
|
|
18
|
+
withheld?: boolean;
|
|
17
19
|
block?: number;
|
|
18
20
|
recordedAt?: string;
|
|
19
21
|
confidenceBps?: number;
|
|
@@ -34,8 +36,13 @@ interface Brief {
|
|
|
34
36
|
/** Index into `levels` of the lowest level that still gets paid. */
|
|
35
37
|
passIndex: number;
|
|
36
38
|
}
|
|
39
|
+
export declare const clean: (text: string) => string;
|
|
37
40
|
export declare function errorMessage(error: unknown): string;
|
|
38
41
|
/** Decision ids about one task on the set's board, oldest first. */
|
|
39
42
|
export declare function decisionsOf({ client, network, set }: Context, task: bigint): Promise<bigint[]>;
|
|
40
43
|
export declare function verifyDecision(ctx: Context, id: bigint): Promise<DecisionReport>;
|
|
44
|
+
export type Outcome = "match" | "withheld" | "unverified" | "mismatch";
|
|
45
|
+
/** Disagreement first, then what could not be read. A withheld record never reads as a match: its hashes were not checked. */
|
|
46
|
+
export declare function outcomeOf(reports: DecisionReport[]): Outcome;
|
|
47
|
+
export declare const WITHHELD_NOT_CHECKED = "This copy is served without the work, so its hashes could not be checked and nothing ties its text to the chain. Its answer and confidence agree with the chain, and for a task so do its brief, levels and threshold. The rest of it, such as the question and the second grader, is the host's word.";
|
|
41
48
|
export {};
|
package/dist/verify.js
CHANGED
|
@@ -1,8 +1,15 @@
|
|
|
1
|
+
import { lookup as dnsLookup, promises as dns } from "node:dns";
|
|
2
|
+
import { Agent as HttpAgent, request as httpRequest } from "node:http";
|
|
3
|
+
import { Agent as HttpsAgent, request as httpsRequest } from "node:https";
|
|
4
|
+
import { BlockList, isIP } from "node:net";
|
|
5
|
+
import { setTimeout as sleep } from "node:timers/promises";
|
|
1
6
|
import { BaseError, decodeFunctionData, encodeAbiParameters, keccak256, pad, toBytes, toHex, } from "viem";
|
|
2
7
|
import { decisionExecutorAbi, decisionLogAbi, modelRegistryAbi, taskBoardAbi } from "./abi.js";
|
|
3
8
|
import { canonicalStringify } from "./canonical.js";
|
|
4
9
|
const Kind = { Hire: 1, Grade: 2, Ruling: 3, Change: 4 };
|
|
5
10
|
const KINDS = { 1: "hire", 2: "grade", 3: "ruling", 4: "change" };
|
|
11
|
+
const Status = { Graded: 4, Disputed: 5, Paid: 6 };
|
|
12
|
+
const STATUS = ["none", "open", "hired", "delivered", "graded", "disputed", "paid", "refunded", "ruled", "appealed"];
|
|
6
13
|
const YES = `0x${"0".repeat(63)}1`;
|
|
7
14
|
const NO = `0x${"0".repeat(64)}`;
|
|
8
15
|
const YES_ANSWERS = new Set(["accept", "pay", "adopt", "allow"]);
|
|
@@ -20,11 +27,28 @@ const named = (answer) => (ADDRESS.test(answer) ? short(answer) : answer);
|
|
|
20
27
|
const percent = (bps) => `${(bps / 100).toFixed(bps % 100 === 0 ? 0 : 2)}%`;
|
|
21
28
|
const utc = (seconds) => new Date(Number(seconds) * 1000).toISOString().replace("T", " ").slice(0, 19);
|
|
22
29
|
const same = (a, b) => a !== undefined && b !== undefined && canonicalStringify(a) === canonicalStringify(b);
|
|
30
|
+
// Posters write brief locations and hosts write records, and all of it reaches a terminal through these messages.
|
|
31
|
+
export const clean = (text) => text.replace(/[\p{Cc}\p{Cf}\u2028\u2029]/gu, "?");
|
|
23
32
|
const display = (uri) => (uri.length > 120 ? `${uri.slice(0, 117)}…` : uri);
|
|
33
|
+
// fetch says only "fetch failed" and keeps the reason, a refused connection or
|
|
34
|
+
// a redirect, in its causes. viem's own errors carry theirs in `details`.
|
|
35
|
+
function rootCause(error) {
|
|
36
|
+
let reason;
|
|
37
|
+
for (let next = error.cause; next instanceof Error; next = next.cause) {
|
|
38
|
+
if (!(next instanceof BaseError))
|
|
39
|
+
reason = next.message || next.code || reason;
|
|
40
|
+
}
|
|
41
|
+
return reason;
|
|
42
|
+
}
|
|
24
43
|
export function errorMessage(error) {
|
|
25
|
-
if (error instanceof
|
|
26
|
-
return
|
|
27
|
-
|
|
44
|
+
if (!(error instanceof Error))
|
|
45
|
+
return clean(String(error));
|
|
46
|
+
const reason = rootCause(error);
|
|
47
|
+
if (error instanceof BaseError) {
|
|
48
|
+
const detail = reason ?? error.details;
|
|
49
|
+
return clean(detail ? `${error.shortMessage} (${detail})` : error.shortMessage);
|
|
50
|
+
}
|
|
51
|
+
return clean(reason ? `${error.message}: ${reason}` : error.message);
|
|
28
52
|
}
|
|
29
53
|
function parseJson(text) {
|
|
30
54
|
try {
|
|
@@ -42,20 +66,24 @@ function decodeText(bytes) {
|
|
|
42
66
|
return undefined;
|
|
43
67
|
}
|
|
44
68
|
}
|
|
45
|
-
/** Base
|
|
46
|
-
const ACP_CHAINS = [8453n, 84532n];
|
|
47
|
-
/**
|
|
69
|
+
/** Base, Base Sepolia, Robinhood Chain and its testnet, where ACP evaluations come from. The service's ACP_CHAINS must stay within this list. */
|
|
70
|
+
const ACP_CHAINS = [8453n, 84532n, 4663n, 46630n];
|
|
71
|
+
/** What a grade's subject is, if it is the one the service derives for work delivered elsewhere: a paid grade by its id, or an ACP evaluation by its job. */
|
|
48
72
|
function outsideGrade(recorded, subject) {
|
|
49
73
|
if (!/^\d+$/.test(recorded))
|
|
50
|
-
return
|
|
74
|
+
return undefined;
|
|
51
75
|
const id = BigInt(recorded);
|
|
52
76
|
if (id >> 256n !== 0n)
|
|
53
|
-
return
|
|
54
|
-
const
|
|
55
|
-
if (
|
|
56
|
-
return
|
|
57
|
-
|
|
77
|
+
return undefined;
|
|
78
|
+
const hex = toHex(id, { size: 32 });
|
|
79
|
+
if (keccak256(encodeAbiParameters([{ type: "string" }, { type: "bytes32" }], ["x402-grade", hex])) === subject)
|
|
80
|
+
return { rule: "x402-grade", id: hex };
|
|
81
|
+
const chain = ACP_CHAINS.find((c) => keccak256(encodeAbiParameters([{ type: "string" }, { type: "uint256" }, { type: "uint256" }], ["acp-evaluation", c, id])) === subject);
|
|
82
|
+
return chain === undefined ? undefined : { rule: "acp-evaluation", chain, job: id };
|
|
58
83
|
}
|
|
84
|
+
const outsideSubject = (outside) => outside.rule === "x402-grade"
|
|
85
|
+
? `subject: a paid grade of work delivered outside BrainChain. keccak256(abi.encode("x402-grade", ${short(outside.id)})), from the record's number, is the subject on chain, so no task on this board`
|
|
86
|
+
: `subject: an evaluation of ACP job ${outside.job} on chain ${outside.chain}. keccak256(abi.encode("acp-evaluation", ${outside.chain}, ${outside.job})), from the record's number, is the subject on chain, so no task on this board`;
|
|
59
87
|
/** TaskBoard.subjectOf(id): the task on this board on this chain. */
|
|
60
88
|
function taskSubject(chainId, board, task) {
|
|
61
89
|
return keccak256(encodeAbiParameters([{ type: "uint256" }, { type: "address" }, { type: "uint256" }], [BigInt(chainId), board, task]));
|
|
@@ -112,47 +140,158 @@ function parseBrief(text) {
|
|
|
112
140
|
throw new Error("the brief is not a title, a text, quality levels and a pass level");
|
|
113
141
|
return b;
|
|
114
142
|
}
|
|
143
|
+
const ATTEMPT_MS = 15_000;
|
|
144
|
+
// One slow try and most of another: a host cannot hold the tool longer, however slowly it sends.
|
|
145
|
+
const FETCH_MS = 30_000;
|
|
146
|
+
// A connection that stalls is given up on soon enough for all three tries to fit in FETCH_MS.
|
|
147
|
+
const STALL_MS = 8_000;
|
|
148
|
+
// A record holds a result of up to 100,000 bytes, quoted twice over, beside its brief and the graders' answers.
|
|
149
|
+
const MAX_RECORD_BYTES = 4 * 1024 * 1024;
|
|
150
|
+
// What a worker will read of a brief.
|
|
151
|
+
const MAX_BRIEF_BYTES = 256 * 1024;
|
|
152
|
+
/** A location the tool will not read. Trying again would not help. */
|
|
153
|
+
class Refused extends Error {
|
|
154
|
+
}
|
|
155
|
+
const PRIVATE = new BlockList();
|
|
156
|
+
for (const [net, bits] of [["0.0.0.0", 8], ["10.0.0.0", 8], ["100.64.0.0", 10], ["127.0.0.0", 8], ["169.254.0.0", 16], ["172.16.0.0", 12], ["192.0.0.0", 24], ["192.168.0.0", 16], ["224.0.0.0", 4], ["240.0.0.0", 4]]) {
|
|
157
|
+
PRIVATE.addSubnet(net, bits, "ipv4");
|
|
158
|
+
}
|
|
159
|
+
for (const [net, bits] of [["::", 128], ["::1", 128], ["64:ff9b::", 96], ["2002::", 16], ["fc00::", 7], ["fe80::", 10], ["fec0::", 10], ["ff00::", 8]]) {
|
|
160
|
+
PRIVATE.addSubnet(net, bits, "ipv6");
|
|
161
|
+
}
|
|
162
|
+
const isPrivate = (address) => PRIVATE.check(address, isIP(address) === 6 ? "ipv6" : "ipv4");
|
|
163
|
+
const bareHost = (url) => url.hostname.replace(/^\[|\]$/g, "");
|
|
164
|
+
/** Whether the RPC this client reads from is on a private address, by address or by what its name resolves to. */
|
|
165
|
+
async function rpcIsPrivate(url) {
|
|
166
|
+
if (typeof url !== "string" || !URL.canParse(url))
|
|
167
|
+
return false;
|
|
168
|
+
const host = bareHost(new URL(url));
|
|
169
|
+
if (isIP(host) !== 0)
|
|
170
|
+
return isPrivate(host);
|
|
171
|
+
const found = await dns.lookup(host, { all: true }).catch(() => []);
|
|
172
|
+
return found.some((entry) => isPrivate(entry.address));
|
|
173
|
+
}
|
|
174
|
+
const privateRpc = new WeakMap();
|
|
175
|
+
function allowsPrivate(client) {
|
|
176
|
+
let known = privateRpc.get(client);
|
|
177
|
+
if (!known) {
|
|
178
|
+
known = rpcIsPrivate(client.transport?.url);
|
|
179
|
+
privateRpc.set(client, known);
|
|
180
|
+
}
|
|
181
|
+
return known;
|
|
182
|
+
}
|
|
183
|
+
const privateAddress = (where, address) => new Refused(`${display(where)} is on the private address ${address}, so it was not fetched`);
|
|
184
|
+
// Checked as the socket connects, so a name cannot pass here and then resolve somewhere else.
|
|
185
|
+
function publicLookup(hostname, options, callback) {
|
|
186
|
+
dnsLookup(hostname, { ...options, all: true }, (error, found) => {
|
|
187
|
+
if (error)
|
|
188
|
+
return callback(error);
|
|
189
|
+
const blocked = found.find((entry) => isPrivate(entry.address));
|
|
190
|
+
if (blocked)
|
|
191
|
+
return callback(privateAddress(hostname, blocked.address));
|
|
192
|
+
if (options.all)
|
|
193
|
+
return callback(null, found);
|
|
194
|
+
return callback(null, found[0]?.address, found[0]?.family);
|
|
195
|
+
});
|
|
196
|
+
}
|
|
197
|
+
// Connections are reused, as fetch does: new ones to a host are what stall.
|
|
198
|
+
const agents = new Map();
|
|
199
|
+
function agentFor(protocol, allowPrivate) {
|
|
200
|
+
const key = `${protocol}${allowPrivate}`;
|
|
201
|
+
let agent = agents.get(key);
|
|
202
|
+
if (!agent) {
|
|
203
|
+
const options = { keepAlive: true, ...(allowPrivate ? {} : { lookup: publicLookup }) };
|
|
204
|
+
agent = protocol === "https:" ? new HttpsAgent(options) : new HttpAgent(options);
|
|
205
|
+
agents.set(key, agent);
|
|
206
|
+
}
|
|
207
|
+
return agent;
|
|
208
|
+
}
|
|
209
|
+
function readOnce(uri, url, wanted, signal) {
|
|
210
|
+
return new Promise((resolve, reject) => {
|
|
211
|
+
const send = url.protocol === "https:" ? httpsRequest : httpRequest;
|
|
212
|
+
// Redirects are not followed: the brain reads without following them, so the location on chain is the one read here too.
|
|
213
|
+
const request = send(url, { signal, agent: agentFor(url.protocol, wanted.allowPrivate), timeout: STALL_MS, headers: { "user-agent": "brainchain-verify" } }, (response) => {
|
|
214
|
+
const chunks = [];
|
|
215
|
+
let size = 0;
|
|
216
|
+
response.on("data", (chunk) => {
|
|
217
|
+
size += chunk.length;
|
|
218
|
+
if (size > wanted.limit)
|
|
219
|
+
request.destroy(new Refused(`${display(uri)} is larger than the ${Math.round(wanted.limit / 1024)} KB a ${wanted.what} may be, so it was not read`));
|
|
220
|
+
else
|
|
221
|
+
chunks.push(chunk);
|
|
222
|
+
});
|
|
223
|
+
response.on("end", () => resolve({
|
|
224
|
+
status: response.statusCode ?? 0,
|
|
225
|
+
bytes: new Uint8Array(Buffer.concat(chunks)),
|
|
226
|
+
withheld: response.headers["x-brainchain-withheld"] === "result",
|
|
227
|
+
}));
|
|
228
|
+
response.on("error", reject);
|
|
229
|
+
});
|
|
230
|
+
request.on("timeout", () => request.destroy(new Error(`${display(uri)} went ${STALL_MS / 1000} seconds without answering`)));
|
|
231
|
+
request.on("error", reject);
|
|
232
|
+
request.end();
|
|
233
|
+
});
|
|
234
|
+
}
|
|
115
235
|
// A records host answers 5xx while it restarts, so those get two more tries.
|
|
116
|
-
async function get(url) {
|
|
236
|
+
async function get(uri, url, wanted, deadline) {
|
|
117
237
|
for (let attempt = 1;; attempt++) {
|
|
118
238
|
try {
|
|
119
|
-
|
|
120
|
-
const response = await fetch(url, { redirect: "error", signal: AbortSignal.timeout(15_000) });
|
|
239
|
+
const response = await readOnce(uri, url, wanted, AbortSignal.any([deadline, AbortSignal.timeout(ATTEMPT_MS)]));
|
|
121
240
|
const transient = response.status === 408 || response.status === 429 || response.status >= 500;
|
|
122
241
|
if (!transient || attempt === 3)
|
|
123
242
|
return response;
|
|
124
243
|
}
|
|
125
244
|
catch (error) {
|
|
126
|
-
if (attempt === 3)
|
|
245
|
+
if (error instanceof Refused || attempt === 3)
|
|
127
246
|
throw error;
|
|
128
247
|
}
|
|
129
|
-
await
|
|
248
|
+
await sleep(attempt * 1_000, undefined, { signal: deadline });
|
|
130
249
|
}
|
|
131
250
|
}
|
|
132
|
-
async function
|
|
251
|
+
async function readBytes(uri, url, wanted, deadline) {
|
|
252
|
+
const response = await get(uri, url, wanted, deadline);
|
|
253
|
+
if (response.status === 404)
|
|
254
|
+
throw new Error(`${display(uri)} answered with status 404: the host no longer has this ${wanted.what}, so trying again will not help`);
|
|
255
|
+
if (response.status < 200 || response.status >= 300)
|
|
256
|
+
throw new Error(`${display(uri)} answered with status ${response.status}`);
|
|
257
|
+
return { bytes: response.bytes, withheld: response.withheld };
|
|
258
|
+
}
|
|
259
|
+
async function fetchBytes(uri, wanted) {
|
|
133
260
|
if (uri.startsWith("data:")) {
|
|
134
261
|
const comma = uri.indexOf(",");
|
|
135
262
|
if (comma < 0 || !uri.slice(0, comma).endsWith(";base64"))
|
|
136
263
|
throw new Error("only base64 data URIs are supported");
|
|
137
|
-
|
|
264
|
+
const bytes = Uint8Array.from(Buffer.from(uri.slice(comma + 1), "base64"));
|
|
265
|
+
if (bytes.length > wanted.limit)
|
|
266
|
+
throw new Refused(`the ${wanted.what} is larger than the ${Math.round(wanted.limit / 1024)} KB it may be, so it was not read`);
|
|
267
|
+
return { bytes, withheld: false };
|
|
138
268
|
}
|
|
269
|
+
if (!URL.canParse(uri))
|
|
270
|
+
throw new Refused(`${display(uri)} is not a URL`);
|
|
139
271
|
const url = new URL(uri);
|
|
140
272
|
if (url.protocol !== "https:" && url.protocol !== "http:")
|
|
141
|
-
throw new
|
|
142
|
-
|
|
143
|
-
if (
|
|
144
|
-
throw
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
273
|
+
throw new Refused(`${url.protocol} URIs are not supported`);
|
|
274
|
+
// A literal address never reaches the lookup below.
|
|
275
|
+
if (!wanted.allowPrivate && isIP(bareHost(url)) !== 0 && isPrivate(bareHost(url)))
|
|
276
|
+
throw privateAddress(uri, bareHost(url));
|
|
277
|
+
const started = Date.now();
|
|
278
|
+
const deadline = AbortSignal.timeout(FETCH_MS);
|
|
279
|
+
// Raced as well as passed down, so no stage of the transport can outlast the deadline.
|
|
280
|
+
const expired = new Promise((_, reject) => deadline.addEventListener("abort", () => reject(deadline.reason), { once: true }));
|
|
281
|
+
try {
|
|
282
|
+
return await Promise.race([readBytes(uri, url, wanted, deadline), expired]);
|
|
283
|
+
}
|
|
284
|
+
catch (error) {
|
|
285
|
+
if (!(error instanceof Refused) && (deadline.aborted || ["TimeoutError", "AbortError"].includes(error.name))) {
|
|
286
|
+
throw new Error(`${display(uri)} did not finish answering within ${Math.round((Date.now() - started) / 1000)} seconds`);
|
|
287
|
+
}
|
|
288
|
+
throw error;
|
|
289
|
+
}
|
|
151
290
|
}
|
|
152
291
|
class Checks {
|
|
153
292
|
list = [];
|
|
154
293
|
add(name, status, message) {
|
|
155
|
-
this.list.push({ name, status, message });
|
|
294
|
+
this.list.push({ name, status, message: clean(message) });
|
|
156
295
|
}
|
|
157
296
|
check(name, pass, ok, otherwise) {
|
|
158
297
|
this.add(name, pass ? "ok" : "fail", pass ? ok : otherwise);
|
|
@@ -161,6 +300,9 @@ class Checks {
|
|
|
161
300
|
function readDecision({ client, set }, id) {
|
|
162
301
|
return client.readContract({ address: set.log, abi: decisionLogAbi, functionName: "get", args: [id] });
|
|
163
302
|
}
|
|
303
|
+
/** Bytes that are not what the chain committed to: a disagreement, where an unreachable host is only a check that could not run. */
|
|
304
|
+
class BriefMismatch extends Error {
|
|
305
|
+
}
|
|
164
306
|
async function briefOf({ client, set }, task) {
|
|
165
307
|
const [posted] = await client.getContractEvents({
|
|
166
308
|
address: set.board,
|
|
@@ -172,9 +314,9 @@ async function briefOf({ client, set }, task) {
|
|
|
172
314
|
});
|
|
173
315
|
if (!posted)
|
|
174
316
|
throw new Error(`task ${task} has no Posted event on board ${set.board}`);
|
|
175
|
-
const { bytes } = await fetchBytes(posted.args.briefURI);
|
|
317
|
+
const { bytes } = await fetchBytes(posted.args.briefURI, { what: "brief", limit: MAX_BRIEF_BYTES, allowPrivate: await allowsPrivate(client) });
|
|
176
318
|
if (keccak256(bytes) !== posted.args.briefHash.toLowerCase())
|
|
177
|
-
throw new
|
|
319
|
+
throw new BriefMismatch(`task ${task}'s brief does not match its briefHash on chain`);
|
|
178
320
|
return parseBrief(decodeText(bytes));
|
|
179
321
|
}
|
|
180
322
|
/** Decision ids about one task on the set's board, oldest first. */
|
|
@@ -215,7 +357,7 @@ export async function verifyDecision(ctx, id) {
|
|
|
215
357
|
}
|
|
216
358
|
const e = recorded.args;
|
|
217
359
|
report.block = Number(recorded.blockNumber);
|
|
218
|
-
report.payloadURI = e.payloadURI;
|
|
360
|
+
report.payloadURI = clean(e.payloadURI);
|
|
219
361
|
c.check("read", e.kind === decision.kind &&
|
|
220
362
|
e.subject === decision.subject &&
|
|
221
363
|
e.inputHash === decision.inputHash &&
|
|
@@ -223,7 +365,7 @@ export async function verifyDecision(ctx, id) {
|
|
|
223
365
|
e.confidenceBps === decision.confidenceBps, `read from DecisionLog ${short(set.log)}: ${kind}, recorded ${report.recordedAt} in block ${recorded.blockNumber}, confidence ${decision.confidenceBps} bps`, `the Recorded event in block ${recorded.blockNumber} differs from what DecisionLog.get(${id}) returns`);
|
|
224
366
|
let fetched;
|
|
225
367
|
try {
|
|
226
|
-
fetched = await fetchBytes(e.payloadURI);
|
|
368
|
+
fetched = await fetchBytes(e.payloadURI, { what: "record", limit: MAX_RECORD_BYTES, allowPrivate: await allowsPrivate(client) });
|
|
227
369
|
}
|
|
228
370
|
catch (error) {
|
|
229
371
|
c.add("fetch", "error", `could not fetch the record: ${errorMessage(error)}`);
|
|
@@ -235,29 +377,35 @@ export async function verifyDecision(ctx, id) {
|
|
|
235
377
|
c.add("fetch", "fail", `${display(e.payloadURI)} is not a decision record`);
|
|
236
378
|
return report;
|
|
237
379
|
}
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
380
|
+
const outside = decision.kind === Kind.Grade ? outsideGrade(record.subject, decision.subject) : undefined;
|
|
381
|
+
const task = !outside && decision.kind !== Kind.Change && /^\d+$/.test(record.subject) ? BigInt(record.subject) : undefined;
|
|
382
|
+
if (task !== undefined)
|
|
383
|
+
report.task = Number(task);
|
|
384
|
+
if (outside)
|
|
385
|
+
report.outside = true;
|
|
386
|
+
const hash = keccak256(fetched.bytes);
|
|
387
|
+
// The header is only the host's word. It counts when the record misses its hash and the chain shows the work may be withheld.
|
|
388
|
+
const excuse = fetched.withheld && hash !== e.payloadHash ? await withholding(ctx, decision, record, task, outside, recorded.blockNumber) : undefined;
|
|
389
|
+
if (excuse?.allowed)
|
|
390
|
+
report.withheld = true;
|
|
391
|
+
c.add("fetch", "ok", `fetched the record from ${display(e.payloadURI)}${excuse?.allowed ? ", served without the work" : ""}`);
|
|
392
|
+
if (excuse)
|
|
393
|
+
c.add("withheld", excuse.allowed ? "ok" : "fail", `withheld: ${excuse.why}`);
|
|
394
|
+
if (excuse?.allowed) {
|
|
395
|
+
c.add("record-hash", "skip", "record hash: this copy is served without the work, so it cannot match payloadHash, and nothing ties its text to the chain");
|
|
241
396
|
c.add("input-hash", "skip", "request hash: the request quotes the withheld work, so it cannot match inputHash");
|
|
242
397
|
}
|
|
243
398
|
else {
|
|
244
|
-
const hash = keccak256(fetched.bytes);
|
|
245
399
|
c.check("record-hash", hash === e.payloadHash, `record hash matches payloadHash ${short(hash)}`, `record hash ${short(hash)} differs from payloadHash ${short(e.payloadHash)}`);
|
|
246
400
|
const input = keccak256(toBytes(canonicalStringify(record.request)));
|
|
247
401
|
c.check("input-hash", input === decision.inputHash, `request hash matches inputHash ${short(input)}`, `request hash ${short(input)} differs from inputHash ${short(decision.inputHash)}`);
|
|
248
402
|
}
|
|
249
403
|
c.check("kind", record.kind === kind, `kind: ${record.kind}`, `the record is a ${record.kind}; the chain recorded a ${kind}`);
|
|
250
|
-
const outside = decision.kind === Kind.Grade && outsideGrade(record.subject, decision.subject);
|
|
251
|
-
const task = !outside && decision.kind !== Kind.Change && /^\d+$/.test(record.subject) ? BigInt(record.subject) : undefined;
|
|
252
|
-
if (task !== undefined)
|
|
253
|
-
report.task = Number(task);
|
|
254
|
-
if (outside)
|
|
255
|
-
report.outside = true;
|
|
256
404
|
if (decision.kind === Kind.Change) {
|
|
257
405
|
c.check("subject", record.subject.toLowerCase() === decision.subject, `subject: ${short(decision.subject)}, the exact call this change names`, "the record's subject differs from the one on chain");
|
|
258
406
|
}
|
|
259
407
|
else if (outside) {
|
|
260
|
-
c.add("subject", "ok",
|
|
408
|
+
c.add("subject", "ok", outsideSubject(outside));
|
|
261
409
|
}
|
|
262
410
|
else {
|
|
263
411
|
c.check("subject", task !== undefined && taskSubject(network.chainId, set.board, task) === decision.subject, `subject: task ${record.subject} on board ${short(set.board)}, chain ${network.chainId}`, `the record names task ${record.subject}, which is not the subject recorded on chain`);
|
|
@@ -301,13 +449,53 @@ export async function verifyDecision(ctx, id) {
|
|
|
301
449
|
await checkTaskThreshold(ctx, c, id, task, decision, record);
|
|
302
450
|
}
|
|
303
451
|
if (outside) {
|
|
304
|
-
checkOutsideGrade(c, record);
|
|
452
|
+
checkOutsideGrade(c, record, report.withheld === true);
|
|
305
453
|
c.add("threshold", "skip", "threshold: no task on this chain, so no board applied this grade; the caller received the verdict");
|
|
306
454
|
}
|
|
307
455
|
if (decision.kind === Kind.Change)
|
|
308
456
|
await checkChange(ctx, c, id, decision, record, state);
|
|
309
457
|
return report;
|
|
310
458
|
}
|
|
459
|
+
const WITHHELD = "[withheld until the work is accepted]";
|
|
460
|
+
const withheldField = (value) => value === undefined || value === WITHHELD;
|
|
461
|
+
/**
|
|
462
|
+
* Whether a record that misses its hash may really be the withheld form the
|
|
463
|
+
* records service serves. The rules are that service's own (access and
|
|
464
|
+
* resultIsPublic in the brain's records.ts), read here from the chain: only
|
|
465
|
+
* grades and rulings carry work, a task's work stays private until it is
|
|
466
|
+
* accepted or paid, a paid grade's until it is accepted, and an ACP
|
|
467
|
+
* evaluation's delivery always.
|
|
468
|
+
*/
|
|
469
|
+
async function withholding(ctx, decision, record, task, outside, block) {
|
|
470
|
+
const { client, network, set } = ctx;
|
|
471
|
+
const refuse = (why) => ({ allowed: false, why: `the host says this copy leaves out the work, but ${why}` });
|
|
472
|
+
if (decision.kind !== Kind.Grade && decision.kind !== Kind.Ruling)
|
|
473
|
+
return refuse(`a ${KINDS[decision.kind] ?? "decision"} record carries no work to leave out`);
|
|
474
|
+
const state = (parseJson(record.request.state) ?? {});
|
|
475
|
+
if (state.result !== WITHHELD || !withheldField(state.execution?.output) || !withheldField(state.execution?.error)) {
|
|
476
|
+
return refuse("the request does not mark the work as withheld where the service does");
|
|
477
|
+
}
|
|
478
|
+
if (outside?.rule === "acp-evaluation")
|
|
479
|
+
return { allowed: true, why: "the delivery in an ACP evaluation is never published" };
|
|
480
|
+
if (outside) {
|
|
481
|
+
if (decision.answer !== YES)
|
|
482
|
+
return { allowed: true, why: `a paid grade is published only once accepted, and Jev answered ${named(record.answer)}` };
|
|
483
|
+
const config = await client.readContract({ address: set.board, abi: taskBoardAbi, functionName: "config", blockNumber: block }).catch(() => client.readContract({ address: set.board, abi: taskBoardAbi, functionName: "config" }));
|
|
484
|
+
const bar = config[1];
|
|
485
|
+
if (decision.confidenceBps >= bar)
|
|
486
|
+
return refuse(`this paid grade cleared the ${percent(bar)} bar, so its record is public`);
|
|
487
|
+
return { allowed: true, why: `a paid grade is published only once accepted, and ${percent(decision.confidenceBps)} is below the ${percent(bar)} bar` };
|
|
488
|
+
}
|
|
489
|
+
if (task === undefined || taskSubject(network.chainId, set.board, task) !== decision.subject)
|
|
490
|
+
return refuse("the record does not name the task on chain");
|
|
491
|
+
const on = await client.readContract({ address: set.board, abi: taskBoardAbi, functionName: "get", args: [task] });
|
|
492
|
+
const house = on.poster.toLowerCase() === set.executor.toLowerCase();
|
|
493
|
+
const status = STATUS[on.status] ?? `status ${on.status}`;
|
|
494
|
+
const open = house ? on.status >= Status.Graded : on.status === Status.Paid || ((on.status === Status.Graded || on.status === Status.Disputed) && on.accepted);
|
|
495
|
+
if (open)
|
|
496
|
+
return refuse(`task ${task} is ${status}${on.accepted ? " and accepted" : ""}, so its record is public`);
|
|
497
|
+
return { allowed: true, why: `task ${task} is ${status}${on.status >= Status.Graded && !on.accepted ? " and its work was not accepted" : ""}, so its work is not public` };
|
|
498
|
+
}
|
|
311
499
|
async function checkBrief(ctx, c, task, decision, record, state) {
|
|
312
500
|
let pending = ctx.briefs.get(task);
|
|
313
501
|
if (!pending) {
|
|
@@ -319,7 +507,7 @@ async function checkBrief(ctx, c, task, decision, record, state) {
|
|
|
319
507
|
brief = await pending;
|
|
320
508
|
}
|
|
321
509
|
catch (error) {
|
|
322
|
-
c.add("brief", "skip", `brief: ${errorMessage(error)}`);
|
|
510
|
+
c.add("brief", error instanceof BriefMismatch ? "fail" : "skip", `brief: ${errorMessage(error)}`);
|
|
323
511
|
if (decision.kind === Kind.Grade)
|
|
324
512
|
c.add("confidence-recomputed", "skip", "grade confidence: needs the brief's pass level");
|
|
325
513
|
return;
|
|
@@ -339,14 +527,22 @@ async function checkBrief(ctx, c, task, decision, record, state) {
|
|
|
339
527
|
}
|
|
340
528
|
checkGradeConfidence(c, record, ascending.map((_, i) => String(i)).slice(ascending.length - 1 - brief.passIndex));
|
|
341
529
|
}
|
|
342
|
-
// No brief on chain to compare with
|
|
343
|
-
function checkOutsideGrade(c, record) {
|
|
530
|
+
// No brief on chain to compare with. payloadHash commits a whole record's levels and pass level, and nothing commits a withheld record's.
|
|
531
|
+
function checkOutsideGrade(c, record, withheld) {
|
|
344
532
|
const levels = record.request.questions.decision.criteria;
|
|
345
|
-
|
|
533
|
+
const { passFrom } = record;
|
|
534
|
+
if (!Array.isArray(levels) || passFrom === undefined) {
|
|
346
535
|
c.add("confidence-recomputed", "skip", "grade confidence: the record does not name its pass level");
|
|
347
536
|
return;
|
|
348
537
|
}
|
|
349
|
-
|
|
538
|
+
// The lowest level never passes, or every result would.
|
|
539
|
+
if (typeof passFrom !== "number" || !Number.isInteger(passFrom) || passFrom < 1 || passFrom >= levels.length) {
|
|
540
|
+
c.add("levels", "fail", `levels: the record's pass level ${JSON.stringify(passFrom)} is not one of levels 1 to ${levels.length - 1} of the ${levels.length} the question lists`);
|
|
541
|
+
c.add("confidence-recomputed", "skip", "grade confidence: needs a valid pass level");
|
|
542
|
+
return;
|
|
543
|
+
}
|
|
544
|
+
c.add("levels", "skip", `levels: the record's own ${levels.length} levels, paid from level ${passFrom}. No brief on chain says what they should be${withheld ? ", and no hash commits them on this copy" : ""}`);
|
|
545
|
+
checkGradeConfidence(c, record, levels.map((_, i) => String(i)).slice(passFrom));
|
|
350
546
|
}
|
|
351
547
|
function checkGradeConfidence(c, record, paidKeys) {
|
|
352
548
|
const parts = paidKeys.map((key) => record.probabilities[key] ?? 0);
|
|
@@ -387,14 +583,17 @@ function checkPaying(c, record, p) {
|
|
|
387
583
|
const confidence = yes ? paying : 1 - paying;
|
|
388
584
|
c.check("confidence-recomputed", Math.abs(confidence - record.confidence) < 1e-6 && record.answer === (yes ? p.yes : p.no), `confidence recomputed: ${steps.join("; ")}, so ${yes ? p.yes : p.no} at ${confidence.toFixed(4)}`, `recomputed under the two-grader rule, the decision is ${yes ? p.yes : p.no} at ${confidence.toFixed(4)}, not ${record.answer} at ${record.confidence}`);
|
|
389
585
|
}
|
|
390
|
-
// Applies each threshold the way TaskBoard.hire, grade and rule do,
|
|
391
|
-
//
|
|
586
|
+
// Applies each threshold the way TaskBoard.hire, grade and rule do, then
|
|
587
|
+
// compares with what the board did. raiseGradeBar can lift the grade and ruling
|
|
588
|
+
// thresholds above what was posted, but only while the task is open, and a task
|
|
589
|
+
// never reopens once hired, so the values read now are the ones any grade or
|
|
590
|
+
// ruling was applied at.
|
|
392
591
|
async function checkTaskThreshold(ctx, c, id, task, decision, record) {
|
|
393
592
|
const { client, set } = ctx;
|
|
394
593
|
const terms = await client.readContract({ address: set.board, abi: taskBoardAbi, functionName: "get", args: [task] });
|
|
395
594
|
const filter = { address: set.board, abi: taskBoardAbi, args: { id: task, decisionId: id }, fromBlock: BigInt(set.fromBlock), strict: true };
|
|
396
595
|
const bps = decision.confidenceBps;
|
|
397
|
-
const rule = (threshold, name) => `threshold: ${named(record.answer)} at ${percent(bps)} is ${bps >= threshold ? "at or above" : "below"}
|
|
596
|
+
const rule = (threshold, name) => `threshold: ${named(record.answer)} at ${percent(bps)} is ${bps >= threshold ? "at or above" : "below"} task ${task}'s ${percent(threshold)} ${name} threshold`;
|
|
398
597
|
if (decision.kind === Kind.Hire) {
|
|
399
598
|
const hires = bps >= terms.hireThresholdBps && decision.answer !== NO;
|
|
400
599
|
const expected = `${rule(terms.hireThresholdBps, "hire")}, so the board ${hires ? "hires it" : "holds the hire"}`;
|
|
@@ -471,3 +670,13 @@ async function checkChange(ctx, c, id, decision, record, state) {
|
|
|
471
670
|
c.add("threshold", "skip", `threshold: ${at} clears today's ${percent(threshold)} change threshold; not queued yet, or a newer decision about the same call replaced it`);
|
|
472
671
|
}
|
|
473
672
|
}
|
|
673
|
+
/** Disagreement first, then what could not be read. A withheld record never reads as a match: its hashes were not checked. */
|
|
674
|
+
export function outcomeOf(reports) {
|
|
675
|
+
const checks = reports.flatMap((r) => r.checks);
|
|
676
|
+
if (checks.some((c) => c.status === "fail"))
|
|
677
|
+
return "mismatch";
|
|
678
|
+
if (checks.some((c) => c.status === "error"))
|
|
679
|
+
return "unverified";
|
|
680
|
+
return reports.some((r) => r.withheld) ? "withheld" : "match";
|
|
681
|
+
}
|
|
682
|
+
export const WITHHELD_NOT_CHECKED = "This copy is served without the work, so its hashes could not be checked and nothing ties its text to the chain. Its answer and confidence agree with the chain, and for a task so do its brief, levels and threshold. The rest of it, such as the question and the second grader, is the host's word.";
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "brainchain-verify",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.4.0",
|
|
4
4
|
"description": "Check a BrainChain decision against the chain: the published record, its hashes, its confidence and the thresholds the contracts applied.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"author": "BrainChain",
|
|
@@ -45,6 +45,7 @@
|
|
|
45
45
|
"scripts": {
|
|
46
46
|
"build": "node scripts/deployments.mjs && tsc -p tsconfig.json",
|
|
47
47
|
"abi": "node scripts/abi.mjs",
|
|
48
|
+
"test": "npm run build && node --test",
|
|
48
49
|
"prepublishOnly": "npm run build"
|
|
49
50
|
},
|
|
50
51
|
"dependencies": {
|