@taifoon/jev 0.1.0 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (49) hide show
  1. package/README.md +97 -191
  2. package/bin/jev.mjs +9 -3
  3. package/bin/run.mjs +86 -0
  4. package/bin/term.mjs +154 -0
  5. package/dist/ask.d.ts +5 -8
  6. package/dist/ask.js +16 -28
  7. package/dist/contracts.d.ts +5 -1
  8. package/dist/contracts.js +7 -3
  9. package/dist/grade.d.ts +0 -4
  10. package/dist/grade.js +5 -6
  11. package/dist/index.d.ts +7 -1
  12. package/dist/index.js +5 -1
  13. package/dist/job.d.ts +65 -0
  14. package/dist/job.js +50 -0
  15. package/dist/pipeline.d.ts +156 -0
  16. package/dist/pipeline.js +193 -0
  17. package/dist/rubric.js +1 -1
  18. package/dist/verify.d.ts +6 -0
  19. package/dist/verify.js +48 -9
  20. package/docs/API.md +20 -0
  21. package/docs/DEVELOP.md +7 -0
  22. package/docs/EVALUATORS.md +20 -0
  23. package/docs/ON_CHAIN.md +44 -0
  24. package/docs/PIPELINE.md +76 -0
  25. package/docs/RECORDS.md +10 -0
  26. package/docs/WORKFLOWS.md +38 -0
  27. package/examples/grade-job.mjs +17 -0
  28. package/examples/jobs/base-bitagent-7287.recorded.json +80 -0
  29. package/examples/jobs/invoice-extraction.json +24 -0
  30. package/examples/jobs/research-report-no-sources.json +29 -0
  31. package/examples/jobs/research-report.json +34 -0
  32. package/examples/jobs/signal-unsupported.json +27 -0
  33. package/examples/quickstart.mjs +22 -0
  34. package/package.json +19 -11
  35. package/workflows/{0.1.0 → 0.2.1}/README.md +3 -3
  36. package/workflows/{0.1.0 → 0.2.1}/jev-grader.json +2 -2
  37. package/workflows/{0.1.0 → 0.2.1}/manifest.json +12 -12
  38. package/workflows/{0.1.0 → 0.2.1}/schemas/answers.json +17 -35
  39. package/workflows/{0.1.0 → 0.2.1}/schemas/common.json +1 -1
  40. package/workflows/{0.1.0 → 0.2.1}/schemas/delivery.json +15 -31
  41. package/workflows/{0.1.0 → 0.2.1}/schemas/facts.json +15 -31
  42. package/workflows/{0.1.0 → 0.2.1}/schemas/hire.json +22 -45
  43. package/workflows/{0.1.0 → 0.2.1}/schemas/job.json +7 -15
  44. package/workflows/{0.1.0 → 0.2.1}/schemas/settlement.json +17 -33
  45. package/workflows/{0.1.0 → 0.2.1}/schemas/verdict.json +17 -35
  46. package/workflows/0.2.1/stamp-grade.json +175 -0
  47. package/workflows/0.1.0/stamp-grade.json +0 -175
  48. /package/workflows/{0.1.0 → 0.2.1}/batch-judge.json +0 -0
  49. /package/workflows/{0.1.0 → 0.2.1}/hire-judge-settle.json +0 -0
package/README.md CHANGED
@@ -1,222 +1,128 @@
1
1
  # @taifoon/jev
2
2
 
3
- Jev is a grader that any protocol can call from a contract's evaluator seat. The pipeline is code facts, then calibrated
4
- answers, then a composed verdict, a receipt, the on-chain record and finally the evaluator call.
3
+ **Did the AI agent do the work it was paid for? Ask Jev, and get an answer you can check.**
5
4
 
6
- Built by Taifoon. MIT. The package has no runtime dependencies and contains no key.
5
+ - **What:** a grader for agent jobs. Code checks the facts, TypeSafe's Jev answers four closed questions about the
6
+ delivery, and code turns the answers into complete, reject or needs review, with a receipt anyone can recompute.
7
+ - **Why:** agents are paid on chain for work nobody reads. A grade with its full probabilities, a published rubric and
8
+ an appeal path is something a buyer, a seller and a contract can all rely on.
9
+ - **How:** replay a real Base job, graded by Jev, offline and with no key:
7
10
 
8
11
  ```
9
- npm i @taifoon/jev
12
+ npx @taifoon/jev run --demo
10
13
  ```
11
14
 
12
- The package also ships the n8n workflows that run Jev as a versioned bundle (see [Workflows](#workflows)).
15
+ Then grade your own with your TypeSafe key from [console.typesafe.ai](https://console.typesafe.ai). No runtime
16
+ dependencies, no key inside, and it signs nothing. Recording a grade on chain is optional.
13
17
 
14
- ## What Jev is
18
+ ## Quick start
15
19
 
16
- Jev is TypeSafe's System One decision model. When this was written it answered as **`jev-1.13.0`**. You don't ask Jev
17
- for prose. You give it closed questions, and every answer comes back as a **full probability distribution** over the
18
- options, plus a confidence value. An answer to "did the delivery meet the spec?" looks like `{ yes: 0.93, no: 0.07 }`,
19
- not a paragraph.
20
-
21
- This package puts Jev inside a fixed judging procedure:
22
-
23
- - Code establishes the facts first. A failed check ends the job, and Jev is never asked.
24
- - Jev answers four **atomic** questions from a published rubric. It never gets a holistic "is the job good?".
25
- - Code composes the verdict from the facts and the answers under published thresholds.
26
- - Everything is hashed into a receipt, recorded on chain and turned into the one call that ends the job.
27
-
28
- How to describe the result: *graded against a published rubric by a pinned decision model, with an appeal.* It is not
29
- "independently verified". An on-chain record is an attestation by whoever sent it, not a proof that the answer is right.
30
-
31
- ## The pipeline
32
-
33
- ```
34
- evidence ─► facts() ──── a check failed? ── yes ─► reject (hard fail; Jev is not asked)
35
- code checks │
36
- no
37
- ▼
38
- Jev reads: evidence (cut at 3,600 chars) + the facts section + one instruction
39
- Jev answers RUBRIC_v1: spec_met · unsupported_claim · ending · cheat_shaped
40
- ▼
41
- compose under THRESHOLDS_v1 ─► complete | reject | needs_review
42
- ▼
43
- receipt: rubric hash, input digest, every distribution, verdict, reasons, receiptHash
44
- ▼
45
- record() ─► JevAnswerLog.record (jev.answer.v1) + JevDecisionLog.record (decision.v2)
46
- ▼
47
- evaluatorCall() ─► complete / reject on the job's protocol (needs_review ends nothing: appeal)
48
- ```
49
-
50
- THRESHOLDS_v1:
51
-
52
- - **complete** needs spec_met ≥ 0.85, unsupported_claim ≤ 0.20 and ending = complete.
53
- - **reject** follows from spec_met ≤ 0.40, unsupported_claim ≥ 0.70 or ending = reject.
54
- - **needs_review** is the result when cheat_shaped ≥ 0.50 (escalate, never slash on this alone) or when the answers
55
- land in the mid band.
56
- - Above 50 USDC a complete is held for review.
57
-
58
- ## Quickstart
59
-
60
- This is `examples/quickstart.mjs`, run as is:
20
+ An agent was hired for a job. The grader answers one question: did it do the work as the task laid it out? You give
21
+ it the job the way the buyer wrote it, and it does the rest:
61
22
 
62
23
  ```js
63
- import { createHash } from 'node:crypto';
64
- import { grade, facts, record, evaluatorCall, verify } from '@taifoon/jev';
65
-
66
- const task = 'Return the SHA-256 hex digest of the ASCII string "jev".';
67
- const delivered = createHash('sha256').update('jev').digest('hex'); // what the seller handed in
68
-
69
- const receipt = await grade({
70
- subject: { chainId: 8453, ref: 'quickstart:sha256-jev' }, // what is being judged (your job id)
71
- evidence: { task, delivered },
72
- facts: facts({ delivered: true, checks: { digest_recomputes: () => createHash('sha256').update('jev').digest('hex') === delivered } }),
73
- trial: true, // 3 free calls; or { key: process.env.TYPESAFE_KEY }
24
+ import { prepareJob, facts, grade, record } from '@taifoon/jev';
25
+
26
+ const job = prepareJob({
27
+ id: 'job-4711', chainId: 8453,
28
+ task: 'Extract the invoice into JSON with invoice_no, total, currency and due_date',
29
+ criteria: ['Returns one JSON object with exactly the four fields', 'Every value matches the source invoice'],
30
+ source: 'INVOICE INV-20931 · Total due: EUR 1,240.50 · Payment due by 2026-10-01',
31
+ delivered: '{"invoice_no":"INV-20931","total":1240.50,"currency":"EUR","due_date":"2026-10-01"}',
32
+ checks: [{ kind: 'json', keys: ['invoice_no', 'total', 'currency', 'due_date'] }],
74
33
  });
75
- console.log(receipt.verdict, '·', receipt.reasons[0]);
76
- console.log('spec_met', receipt.answers?.spec_met.probabilities, '· model', receipt.model);
77
- const onchain = await record(receipt); // unsigned: JevAnswerLog + JevDecisionLog (devnet)
78
- const end = evaluatorCall('virtuals-erc8183', 1n, receipt, receipt); // unsigned: complete / reject as job 1's evaluator
79
- console.log(onchain.calls.map((c) => c.fn), '→', end?.fn, '· verifies:', (await verify(receipt, { chain: false })).ok);
80
- ```
81
-
82
- Real output from one call to the public trial (2026-09-27):
83
-
84
- ```
85
- complete · spec_met 0.93 ≥ 0.85 · unsupported_claim 0.10 ≤ 0.2 · ending complete
86
- spec_met { no: 0.07, yes: 0.93 } · model jev-1.13.0
87
- [ 'JevAnswerLog.record', 'JevDecisionLog.record' ] → complete(uint256,bytes32,bytes) · verifies: true
34
+ const receipt = await grade({ subject: job.subject, evidence: job.evidence, facts: facts(job.facts), key: process.env.TYPESAFE_KEY });
35
+ receipt.verdict; // complete | reject | needs_review, with every answer's full distribution
36
+ await record(receipt); // the unsigned calls that put it on chain
88
37
  ```
89
38
 
90
- ## The five calls
91
-
92
- | Call | What it does | Network |
93
- |---|---|---|
94
- | `grade({ subject, evidence, facts?, rubric?, key? \| trial })` | Checks the facts first. If none fails, asks Jev the rubric, composes the verdict and returns the receipt. | Asks Jev once, or never on a hard fail |
95
- | `facts({ delivered, checks, priceUsdc? })` | Runs the checks your protocol supplies. A check is a value or an (async) function; one that throws counts as "could not be checked". | None |
96
- | `record(receipt, { network, send? })` | Returns the unsigned `JevAnswerLog.record` and `JevDecisionLog.record` calls. `network` is a flag: `none`, `devnet` (default), `base` or `both`. Pass `send` (your wallet) to send every call that has an address. | None unless you pass `send` |
97
- | `evaluatorCall(protocol, jobId, verdict, digest)` | Returns the one unsigned call that ends the job on that protocol. For `needs_review` it returns `null`. | None |
98
- | `verify(receiptOrDigest)` | Recomputes every digest, re-composes the verdict and finds the `JevAnswered` and `Decided` events. | One devnet RPC read |
99
-
100
- It also exports `RUBRIC_v1` and `CONTRACTS`. Nothing else.
101
-
102
- **Where Jev is reached.** With `trial: true`, calls go to `https://typesafe.taifoon.dev/v1/trial`. The trial is free for
103
- 3 calls per caller, takes at most 4 questions and 4,000 JSON characters, and SDK cuts the evidence to fit (never the facts
104
- section). With `key`, calls go to `https://api.typesafe.ai/v1/systemone` using your own TypeSafe key from
105
- [console.typesafe.ai](https://console.typesafe.ai). The key is not stored and never appears in a receipt. If you already
106
- have the answers (from the n8n node, for example), pass `answers` and nothing is asked.
107
-
108
- **Your own rubric.** `grade({ rubric: { version, questions, asked?, thresholds?, compose? } })`. The rubric hash is
109
- sha256 of its questions and goes into the receipt.
39
+ What the grader needs, and where it comes from:
110
40
 
111
- ## Evaluator seats
112
-
113
- Each adapter is one small file with its ABI fragment. Each one is tested against calldata that a mined transaction
114
- carried (`test/fixtures/evaluator-calls.json`, written by `scripts/record-evaluator-fixtures.mjs`).
115
-
116
- | `protocol` | Call | Tested against |
117
- |---|---|---|
118
- | `assurance-hook` | `complete(bytes32)` / `reject(bytes32 jobId, bytes32 digest)`, with `opts.to` = the hook | Base [`0xc606d00f…3116`](https://basescan.org/tx/0xc606d00fed64e912585bbc93feb15a9289cfcde94289f49f7c74a7a468903116), the complete of a covered GLMR hire on Moonbeam's hook (funded in [`0xfbe436ce…f193`](https://basescan.org/tx/0xfbe436ced5caf66dde0e96033f041480316fc5f62cd273006ad9c1852a57f193)) |
119
- | `judge-adapter` | `postVerdict(bytes32 jobId, 1\|2, 2, bytes32 digest)` → hook complete / reject | devnet [`0x07853069…2012`](https://www.taifoon.io/scan/36927/tx/0x07853069cf43200fad3761d05b532c654df6f02b8a47f5f767bec18cb91a2012) |
120
- | `virtuals-erc8183` | AgenticCommerceV3 `complete` / `reject(uint256, bytes32 reason, bytes)` as the job's evaluator | Base [`0x60890802…27bd`](https://basescan.org/tx/0x60890802a3b399af7face54c1cb33d771cffc9b8594d0113de14ae290b5c27bd) (complete) and [`0x152b5fb8…db76`](https://basescan.org/tx/0x152b5fb8900e57119cee6cbc15206d8938c11dc4056b20ec4c986f4d8acddb76) (reject), inside ERC-4337 handleOps |
121
- | `virtuals-memo-acp` | ACPRouter `signMemo(uint256 memoId, bool approved, string reason)` on the evaluation memo | Base [`0x94ba6b49…622e`](https://basescan.org/tx/0x94ba6b497e2fc2ecf7ae256e1a490cabdc4a5357ed306ee2071f46a1c4ef622e) |
122
- | `bitagent-erc8183` | AgenticCommerce `complete` / `reject(uint256, bytes32, bytes)` as the job's evaluator; `seat: 'platform'` → Evaluator `forceSettleJob(uint256, bool)` | Base [`0x195359ba…b01e`](https://basescan.org/tx/0x195359ba64f8ba429c449e82c27d0c57486511593646886b59a0d80c808fb01e) (forceSettleJob, job 8941) and its submit [`0xa4a49696…4ebc`](https://basescan.org/tx/0xa4a496968877ccbf1ba73fe5c5908f064d7333a2ced984e1867da9ee70f74ebc) (same layout) |
123
-
124
- The digest in the call is the receipt's decision digest. That is the same 32 bytes JevDecisionLog holds, so the job's
125
- ending, the decision record and the receipt all carry one value. On a hard fail no decision exists, so pass
126
- `receipt.receiptHash`.
127
-
128
- Whoever holds the seat signs the call; this package never does. The BitAgent implementation is unverified on Base, and
129
- its layout was read from its own transactions. Its jobs settle today through the platform Evaluator, which only
130
- BitAgent's operators can call.
131
-
132
- ## On chain today
133
-
134
- The logs are on the public devnet (chain 36927, free gas from `https://faucet.taifoon.dev`). Any account may send to
135
- them.
136
-
137
- | Contract | Address | Rule |
41
+ | Field | What it is | Who checks it |
138
42
  |---|---|---|
139
- | JevAnswerLog | `0xD8c1d8188Fc8EE388792f6EB3B3dbd4f341Ba8e3` | Append-only, no owner. Recorders fixed at deploy are `trusted`; any other sender is logged with `trusted = false` |
140
- | JevDecisionLog | `0x1D622511862DD7DEffB8fEeBc225E0FAdA1e9A05` | Append-only, no owner, no upgrade. The recorder is `msg.sender` |
141
-
142
- For Base, `CONTRACTS.base.answerLog` and `CONTRACTS.base.decisionLog` are `null` in 0.1.0. So `record(receipt, { network: 'base' })` returns the Base call data with `to: null` and `status: 'partial'`, and `send` skips those calls.
43
+ | `task` | what the buyer asked for, in their words | Jev reads it |
44
+ | `criteria` | what "done" means, one per line | Jev reads them numbered |
45
+ | `source` | what the delivery must be faithful to (an invoice, a page, a dataset) | Jev compares against it |
46
+ | `delivered` | what the agent handed in | code first, then Jev |
47
+ | `checks` | `includes` · `sources` · `json` · `deadline` | code; a failed check rejects before Jev is asked |
143
48
 
144
- Here is one real grade, end to end: n8n execution 86, a proof-verification job graded with RUBRIC_v1.
145
-
146
- - Answers recorded in devnet tx
147
- [`0x9a38cf55…a61b`](https://www.taifoon.io/scan/36927/tx/0x9a38cf5541bb82a8a6e2e5cf1a96ed25fdc46921c351506da83023f5dec1a61b).
148
- The answers digest is `0x6c7e6d2977689f1c9639a377b165f998d1160cc30eca8ff4f1fedc793db2f52d`.
149
- - Decision anchored in devnet tx
150
- [`0x687ba9d7…4ee4`](https://www.taifoon.io/scan/36927/tx/0x687ba9d73d5789a2df9d9d09e94b6ed005447d7f5ece97f9226ea0d51f514ee4).
151
- Decision digest `0x781b0fce…3667`, confidence 2,700 bps.
152
- - Jev answered spec_met `{ yes: 0.36, no: 0.64 }`, so the composed verdict is **reject** (spec_met ≤ 0.40).
153
-
154
- `verify('0x6c7e6d29…f52d')` finds both rows. This is the real output:
49
+ Four illustrative jobs, written by hand, are in [`examples/jobs/`](examples/jobs): a research report that meets the brief, the same report
50
+ without its sources (code rejects it), an invoice extraction checked against its source, and a trading signal whose
51
+ claims the evidence cannot support. Grade one with your key:
155
52
 
156
53
  ```
157
- {"ok":true,"checks":{"answersOnChain":true,"decisionOnChain":true},
158
- "answer":{"tx":"0x9a38cf5541bb82a8a6e2e5cf1a96ed25fdc46921c351506da83023f5dec1a61b","trusted":true},
159
- "decision":{"tx":"0x687ba9d73d5789a2df9d9d09e94b6ed005447d7f5ece97f9226ea0d51f514ee4","confidenceBps":2700}}
54
+ TYPESAFE_KEY=… npx @taifoon/jev run --job-file examples/jobs/invoice-extraction.json
55
+ npx @taifoon/jev run --demo # offline, no key: replays the real Base job below and checks it against the chain
56
+ npx @taifoon/jev run # a live Base job from the coordination layer's queue (needs TYPESAFE_KEY)
160
57
  ```
161
58
 
162
- The tests re-derive these rows offline, byte for byte:
163
-
164
- - **Execution 86:** the answers digest, the `JevAnswerLog.record` calldata (equal to the mined input) and the decision
165
- digest.
166
- - **Execution 88:** run through `grade()` itself, it lands on the anchored decision digest and composes needs_review.
59
+ ## A real job, graded
60
+
61
+ [BitAgent](https://basescan.org/address/0x5009ABB3A309115a4a682C66BAf3BC9E0329BaB7) job 7287 on Base: a buyer paid
62
+ 1.5 USDC for `equity_research where ticker is 'AAPL'`. The seller submitted within seconds, but only a 32-byte digest
63
+ reached the chain; the report itself was never shown. Code confirmed the job was funded and delivered, then Jev
64
+ (`jev-1.13.0`) answered on 27 September 2026:
65
+
66
+ | Question | Jev's answer |
67
+ |---|---|
68
+ | spec_met: does the delivery meet the task? | **no** 0.97 · yes 0.03 |
69
+ | unsupported_claim | no 0.94 · yes 0.06 |
70
+ | ending | **needs_review** 0.93 · complete 0.07 |
71
+ | cheat_shaped | no 0.56 · yes 0.44 |
72
+
73
+ Code composed **reject** (spec_met ≤ 0.40) and the unsigned `reject(uint256,bytes32,bytes)` for BitAgent's evaluator
74
+ seat. The decision is anchored on the Taifoon devnet in
75
+ [`0xffc608f8…c622`](https://www.taifoon.io/scan/36927/tx/0xffc608f8c8b39ff6992e8ec8d589b8784a3cf4199359c408f8eed1d4b581c622).
76
+ `npx @taifoon/jev run --demo` replays it from [`examples/jobs/base-bitagent-7287.recorded.json`](examples/jobs/base-bitagent-7287.recorded.json)
77
+ with no key and no network, and arrives at the same decision digest.
78
+
79
+ ## What you must know
80
+
81
+ - **Code decides the facts first.** A failed check ends the job as reject, and Jev is never asked.
82
+ - **Jev answers four atomic questions** from RUBRIC_v1: spec_met, unsupported_claim, ending and cheat_shaped. It
83
+ never gets a holistic "is the job good?".
84
+ - **Code composes the verdict** under THRESHOLDS_v1: complete, reject or needs_review. needs_review ends nothing and
85
+ leaves room for an appeal. Above 50 USDC a complete is held for review.
86
+ - **Nothing is signed.** `record()` and `evaluatorCall()` return unsigned calls. Whoever holds the seat signs.
87
+ - **Your key.** Jev is called with your own TypeSafe key from [console.typesafe.ai](https://console.typesafe.ai),
88
+ directly at TypeSafe. The key is not stored and never appears in a receipt.
89
+ - **What the result is.** *Graded against a published rubric by a pinned decision model, with an appeal.* It is not
90
+ "independently verified". An on-chain record is an attestation by whoever sent it, not a proof the answer is right.
167
91
 
168
- ## The records
169
-
170
- - **Receipt.** `receiptHash = sha256(JSON.stringify(body))`. The body holds the rubric and its hash, the thresholds, the
171
- subject, `stateHash`, the facts, the model, every answer with its distribution, what was not asked, the verdict,
172
- `auto`, `forced`, the reasons and the scores.
173
- - **decision.v2.** `digest = sha256(canonical { v, kind, subject{chainId, at, ref}, model, answers[id, question,
174
- options, value, probabilities, confidence], input_digest })`, with keys sorted.
175
- `subjectId = keccak256(abi.encode("taifoon.decision.subject.v1", chainId, at, ref))`.
176
- - **jev.answer.v1.** `digest = sha256(canonical record)`, covering questions, answers with distributions, model,
177
- upstream model, latency, credential path (`trial` or `caller-credential`), caller and time.
178
-
179
- ## In n8n
180
-
181
- The TypeSafe node (`@taifoon/n8n-nodes-typesafe` 1.5.0) carries this package's network-free core under **Jev Options**:
182
-
183
- - **Ask RUBRIC_v1** routes Pass = complete, Fail = reject and Review = needs_review.
184
- - **Facts (JSON)** takes checks you already made. A false check rejects without asking Jev.
185
- - **Record On** takes `none`, `devnet`, `base` or `both`, and **Evaluator Call** takes a protocol and a job id.
186
-
187
- The node outputs the receipt and the unsigned calls, and it signs nothing. With the options unset it behaves exactly as
188
- 1.4.0.
189
-
190
- ## Workflows
92
+ ## The five calls
191
93
 
192
- `workflows/0.1.0/` holds the four n8n workflows that run Jev on n8n.taifoon.dev. The same files are attached to the
193
- GitHub release as `jev-workflows-0.1.0.zip`:
94
+ | Call | What it does |
95
+ |---|---|
96
+ | `prepareJob({ id, task, criteria, delivered, source?, checks? })` | Turns a job as the buyer laid it out into the evidence Jev reads and the code checks |
97
+ | `grade({ subject, evidence, facts?, rubric?, key \| answers })` | Checks the facts, asks Jev the rubric, composes the verdict, returns the receipt |
98
+ | `facts({ delivered, checks, priceUsdc? })` | Runs the checks your protocol supplies |
99
+ | `record(receipt, { network, send? })` | Returns the unsigned `JevAnswerLog.record` and `JevDecisionLog.record` calls |
100
+ | `evaluatorCall(protocol, jobId, verdict, digest)` | Returns the one unsigned call that ends the job; `null` for needs_review |
101
+ | `verify(receiptOrDigest)` | Recomputes every digest and finds the records on chain |
194
102
 
195
- - **`hire-judge-settle.json`** runs hire → prepare (facts) → TypeSafe (the four RUBRIC_v1 questions) → answers
196
- (verdict, recorded and anchored) → settle on devnet. Executions 86 and 88 of this workflow are the golden runs above.
197
- - **`jev-grader.json`** grades Base jobs nobody ruled on, then stamps each verdict on the devnet GradeStampRegistry.
198
- - **`batch-judge.json`** runs prepare → Jev → answers for each subject, then `POST /v1/judge/batch`.
199
- - **`stamp-grade.json`** takes a recorded decision, turns it into the canonical verdict and stamps it.
103
+ Evaluator seats: `assurance-hook`, `judge-adapter`, `virtuals-erc8183`, `virtuals-memo-acp` and `bitagent-erc8183`.
104
+ Each is tested against calldata a mined transaction carried.
200
105
 
201
- Credential references are placeholders: `{ "id": "", "name": "REPLACE: <credential type>" }`. After import, pick your
202
- own credential of that type. Each step names the entity it takes and emits by `$id` (`meta.taifoon.steps`). The schemas
203
- behind those `$id`s are in `workflows/0.1.0/schemas/`. `manifest.json` pins every file by sha256.
204
- `workflows/0.1.0/README.md` explains each workflow step by step and lists the nodes and credentials it needs:
106
+ ## Documentation
205
107
 
206
- - `@taifoon/n8n-nodes-typesafe`, tested with 1.3.0, is on npm.
207
- - `n8n-nodes-taifoon`, tested with 0.4.2, and `n8n-nodes-taifoon-devnet-signer`, tested with 0.2.0, are custom
208
- extensions and are not on npm.
108
+ - [How grading works: Jev, the pipeline, THRESHOLDS_v1, `npx @taifoon/jev run`](docs/PIPELINE.md)
109
+ - [API: the five calls, where Jev is reached, your own rubric](docs/API.md)
110
+ - [Evaluator seats, with the transactions they are tested against](docs/EVALUATORS.md)
111
+ - [On chain: contract addresses on Base and devnet, and one real grade end to end](docs/ON_CHAIN.md)
112
+ - [The records: receipt, decision.v2, jev.answer.v1](docs/RECORDS.md)
113
+ - [n8n workflows, and the TypeSafe node's Jev Options](docs/WORKFLOWS.md)
114
+ - [Develop](docs/DEVELOP.md)
209
115
 
210
116
  ```
211
- npx @taifoon/jev workflows list
212
- npx @taifoon/jev workflows export ./jev-workflows # workflows + schemas + manifest + README
213
- npx @taifoon/jev verify 0x6c7e6d2977689f1c9639a377b165f998d1160cc30eca8ff4f1fedc793db2f52d
117
+ npx @taifoon/jev verify <digest> --network base
118
+ npx @taifoon/jev workflows export ./jev-workflows
214
119
  ```
215
120
 
216
- ## Develop
121
+ Pricing the next job from a seller's record is a separate package:
122
+ [`@taifoon/jev-wilson`](https://github.com/taifoon-io/jev-wilson).
217
123
 
218
- ```
219
- npm ci && npm run check # typecheck, tests (golden runs, recorded calldata, hashes vs viem), build
220
- node scripts/vendor-n8n.mjs # refresh the n8n node's copy of the core; test/vendor.test.ts fails when it drifts
221
- npm run bundle && npm run zip # rebuild workflows/<version>/ from the umbrella's workflows/ and zip it for the release
222
- ```
124
+ ## Licence
125
+
126
+ Independent project. Jev and TypeSafe are products of TypeSafe AI, Inc., which does not endorse this package.
127
+
128
+ Built by Taifoon. MIT. The package has no runtime dependencies and contains no key. It calls Jev only with your own TypeSafe key.
package/bin/jev.mjs CHANGED
@@ -2,7 +2,8 @@
2
2
  // jev — the @taifoon/jev command line.
3
3
  // jev workflows list [--version X] the bundled n8n workflows (and which nodes and credentials each needs)
4
4
  // jev workflows export <dir> [--version X] write them, their schemas, manifest and README into <dir>
5
- // jev verify <answers-digest> find the digest's JevAnswered / Decided rows on devnet 36927
5
+ // jev run [--job 8453:81100] [--evidence pack.json] [--network …] [--yes] [--json] one job through the pipeline, step by step
6
+ // jev verify <answers-digest> [--network devnet|base|any] find the digest's JevAnswered / Decided rows (default any: devnet, then Base 8453)
6
7
  import { cpSync, existsSync, readdirSync, readFileSync } from 'node:fs';
7
8
  import { dirname, join, resolve } from 'node:path';
8
9
  import { fileURLToPath } from 'node:url';
@@ -12,6 +13,7 @@ const PKG = JSON.parse(readFileSync(join(ROOT, 'package.json'), 'utf8'));
12
13
  const args = process.argv.slice(2);
13
14
  const flag = (n) => { const i = args.indexOf(n); return i >= 0 ? args.splice(i, 2)[1] : undefined; };
14
15
  const version = flag('--version') ?? PKG.version;
16
+ const network = flag('--network') ?? 'any';
15
17
  const bundle = join(ROOT, 'workflows', version);
16
18
  const die = (m) => { console.error(m); process.exit(1); };
17
19
  const [cmd, sub, target] = args;
@@ -27,11 +29,15 @@ if (cmd === 'workflows') {
27
29
  cpSync(bundle, out, { recursive: true });
28
30
  console.log(`${m.bundle} → ${out}: ${m.workflows.length} workflows, ${m.schemas.length} schemas, manifest.json, README.md`);
29
31
  } else die('usage: jev workflows list|export <dir> [--version X]');
32
+ } else if (cmd === 'run') {
33
+ await import('./run.mjs').then((m) => m.run(process.argv.slice(3)));
30
34
  } else if (cmd === 'verify' && sub) {
31
35
  const { verify } = await import('../dist/index.js');
32
- const v = await verify(sub);
36
+ if (!['devnet', 'base', 'any'].includes(network)) die('--network is devnet, base or any');
37
+ let v = await verify(sub, { network: network === 'base' ? 'base' : 'devnet' });
38
+ if (network === 'any' && !v.checks.answersOnChain) { const b = await verify(sub, { network: 'base' }); if (b.checks.answersOnChain || !v.ok) v = b; }
33
39
  console.log(JSON.stringify(v, null, 1));
34
40
  process.exit(v.ok ? 0 : 2);
35
41
  } else {
36
- console.log(`jev ${PKG.version}\n jev workflows list [--version X]\n jev workflows export <dir> [--version X]\n jev verify <answers-digest>`);
42
+ console.log(`jev ${PKG.version}\n jev workflows list [--version X]\n jev workflows export <dir> [--version X]\n jev run [--job <chain>:<id>] [--evidence pack.json] [--network none|devnet|base|both] [--yes] [--json]\n jev verify <answers-digest> [--network devnet|base|any]`);
37
43
  }
package/bin/run.mjs ADDED
@@ -0,0 +1,86 @@
1
+ // jev run — one job through the pipeline, one step at a time.
2
+ // jev run the first ready job on the coordination layer's queue (needs TYPESAFE_KEY)
3
+ // jev run --job 8453:81100 a job you name (chain:id; ids like bitagent:8453:7287 work too)
4
+ // jev run --evidence pack.json your own pack, no layer at all: { subject, state, delivered?, checks?, priceUsdc? }
5
+ // jev run --job-file job.json one agent job: { id, task, criteria[], delivered, source?, checks[] } (see examples/jobs/)
6
+ // jev run --demo offline, no key: replays a REAL graded job (BitAgent 7287 on Base) with Jev's
7
+ // recorded answers, and checks the result against the decision on chain
8
+ // jev run --answers answers.json answers you already have (the n8n TypeSafe node's output): Jev is not asked
9
+ // --record none|devnet|base|both opt-in: where the grade is recorded (default none: the receipt and digests only);
10
+ // --network is the same flag
11
+ // --layer <url> | --no-layer the coordination layer (default https://coord.taifoon.dev)
12
+ // --yes run every step without asking; --json print the whole trace as JSON at the end
13
+ // Keys come from the environment only: TYPESAFE_KEY (your own TypeSafe key, console.typesafe.ai) and
14
+ // TAIFOON_RELAYER_KEY (records the answers on the layer). Neither is printed or written anywhere.
15
+ // Output is @taifoon/term (vendored as ./term.mjs), the same rhythm as `taifoon up` and the STUDIO run log.
16
+ import { readFileSync } from 'node:fs';
17
+ import { createInterface } from 'node:readline/promises';
18
+ import { createTerm } from './term.mjs';
19
+
20
+ const short = (s, n = 18) => (typeof s === 'string' && s.length > n ? `${s.slice(0, n - 8)}…${s.slice(-6)}` : String(s));
21
+
22
+ /** One line per step: what it established. The full output is in the --json trace. */
23
+ function factOf(r) {
24
+ const o = r.out ?? {};
25
+ switch (r.id) {
26
+ case 'pick': return o.row ? `job ${o.row.jobId} · ${o.row.why} · evidence ${o.row.evidence_ready}` : `job ${o.jobId} on ${o.chainId}`;
27
+ case 'evidence': return `${o.facts} labelled facts · ${o.gaps?.length ?? 0} gaps · ${o.state_chars} characters for Jev to read`;
28
+ case 'facts': return `delivered ${o.delivered ? 'yes' : 'no'} · checks ${o.checksOk === false ? 'failed' : o.checksOk ? 'ok' : 'none'}${o.priceUsdc != null ? ` · ${o.priceUsdc} USDC` : ''}`;
29
+ case 'grade': return `${o.verdict} · ${o.reasons?.[0] ?? ''}${o.model ? ` · ${o.model}` : ''}`;
30
+ case 'record': return `${o.status === 'none' ? 'not recorded (opt in with --record)' : `${o.status} · ${(o.calls ?? []).join(', ')}`}${o.recording?.held ? ` · layer held it: ${o.recording.held}` : ''}${typeof o.layer === 'object' && o.layer ? ` · layer ${short(o.layer.digest)}` : ''}`;
31
+ case 'evaluator': return `${o.fn} on ${o.protocol} · signed by ${o.signer}`;
32
+ case 'premium': return `${o.guaranteed ? 'guaranteed' : 'not guaranteed'} · premium ${o.premium_label ?? o.premium_ratio}`;
33
+ case 'verify': return o.ok ? `the receipt re-derives · ${Object.keys(o.checks ?? {}).length} checks` : `problems: ${(o.problems ?? []).join('; ')}`;
34
+ default: return 'done';
35
+ }
36
+ }
37
+
38
+ export async function run(argv) {
39
+ const a = [...argv];
40
+ const has = (n) => { const i = a.indexOf(n); if (i < 0) return false; a.splice(i, 1); return true; };
41
+ const val = (n) => { const i = a.indexOf(n); return i >= 0 ? a.splice(i, 2)[1] : undefined; };
42
+ const yes = has('--yes') || !process.stdin.isTTY; const json = has('--json'); const noLayer = has('--no-layer') || argv.includes('--demo') || argv.includes('--job-file');
43
+ const demo = has('--demo');
44
+ const job = val('--job'); const evFile = val('--evidence');
45
+ const jobFile = val('--job-file');
46
+ const ansFile = val('--answers');
47
+ const rec = demo ? JSON.parse(readFileSync(new URL('../examples/jobs/base-bitagent-7287.recorded.json', import.meta.url), 'utf8')) : null;
48
+ const network = val('--record') ?? val('--network') ?? 'none'; const layer = noLayer ? false : val('--layer');
49
+ const protocol = val('--protocol'); const price = val('--price-usdc');
50
+ const { pipeline, STEPS, prepareJob } = await import('../dist/index.js');
51
+
52
+ const t = createTerm({ quiet: json });
53
+ const rl = yes ? null : createInterface({ input: process.stdin, output: process.stdout });
54
+ t.info(`jev run · ${noLayer ? 'independent (no layer)' : `layer ${layer ?? 'https://coord.taifoon.dev'}`} · grade on ${demo ? `Jev's recorded answers for ${rec.job.ref} (${rec.job.task})` : ansFile ? 'supplied answers' : process.env.TYPESAFE_KEY ? 'your TypeSafe key' : 'no key: set TYPESAFE_KEY, or try --demo'} · record → ${network === 'none' ? 'none (opt in with --record devnet|base|both)' : network}${process.env.TAIFOON_RELAYER_KEY ? ' + the layer' : ''}`);
55
+
56
+ const trace = await pipeline({
57
+ layer, job, network, protocol, priceUsdc: price ? Number(price) : undefined,
58
+ evidence: rec ? { subject: rec.subject.ref, chainId: rec.subject.chainId, label: rec.subject.label, state: rec.evidence, delivered: rec.facts.delivered, checks: rec.facts.checks } : jobFile ? prepareJob(JSON.parse(readFileSync(jobFile, 'utf8'))).pack : evFile ? JSON.parse(readFileSync(evFile, 'utf8')) : undefined,
59
+ answers: rec ? rec.answers : ansFile ? JSON.parse(readFileSync(ansFile, 'utf8')) : undefined, ...(rec ? { model: rec.model } : {}),
60
+ key: process.env.TYPESAFE_KEY || null, relayerKey: process.env.TAIFOON_RELAYER_KEY || null,
61
+ before: async (s) => {
62
+ t.step(STEPS.indexOf(s) + 1, STEPS.length, s.title);
63
+ if (s.route) t.note(s.route);
64
+ if (!rl) return true;
65
+ const r = (await rl.question(s.required ? ' enter = run · q = quit › ' : ' enter = run · s = skip · q = quit › ')).trim().toLowerCase();
66
+ if (r === 'q') { rl.close(); process.exit(0); }
67
+ return s.required || r !== 's';
68
+ },
69
+ after: (r) => {
70
+ if (r.skipped) t.skip(r.skipped);
71
+ else if (!r.ok) t.fail(r.error ?? 'failed', { ms: r.ms });
72
+ else t.ok(factOf(r), { ms: r.ms });
73
+ },
74
+ });
75
+ rl?.close();
76
+ if (json) t.json(trace);
77
+ else if (trace.receipt) {
78
+ t.line('');
79
+ t.say(`verdict ${trace.receipt.verdict} · receipt ${trace.receipt.receiptHash}`);
80
+ if (rec) {
81
+ const same = trace.receipt.decision?.digest === rec.recorded.digest;
82
+ (same ? t.ok : t.fail)(`${same ? 'same decision digest as' : 'differs from'} the one recorded on chain: ${rec.recorded.decision}, anchored in ${rec.recorded.anchor.tx}`);
83
+ } else if (trace.recorded && trace.recorded.status !== 'none') t.cmd(`npx @taifoon/jev verify ${trace.recorded.digests.answers}`);
84
+ }
85
+ process.exit(trace.steps.some((s) => !s.ok) ? 1 : 0);
86
+ }
package/bin/term.mjs ADDED
@@ -0,0 +1,154 @@
1
+ // @taifoon/term — the step renderer shared by `taifoon up` and `jev run`.
2
+ //
3
+ // One file, no imports, no dependencies, and nothing happens at import time: every env read, TTY check and write
4
+ // is inside a function, so this file can be vendored as-is (a consumer copies it byte for byte and pins its sha256).
5
+ //
6
+ // The rhythm is the STUDIO terminal on taifoon.io:
7
+ // → [1/5] readiness step() bold
8
+ // ● GET coord.taifoon.dev/v1/agents/8453/95902/… call() before the read
9
+ // ⎿ 200 · 612 ms called() after it
10
+ // ✓ hireable · 8 ok · 2 missing · 640 ms ok() / warn() / fail() / skip() ✓ ! ✗ ·
11
+ // the guide's line say() / note()
12
+ // → calibrated: POST /v1/pools/quote next() and its command, indented, via cmd()
13
+ // Colour is on only for a TTY stream without NO_COLOR (FORCE_COLOR overrides both ways) and TERM != dumb.
14
+
15
+ export const GLYPH = Object.freeze({ ok: '✓', warn: '!', fail: '✗', skip: '·' });
16
+ export const SPIN = Object.freeze(['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏']);
17
+ export const MARKS = Object.freeze(['ok', 'warn', 'fail', 'skip']);
18
+
19
+ const wrap = (open, close) => (s) => `\u001b[${open}m${s}\u001b[${close}m`;
20
+ const plain = (s) => String(s);
21
+
22
+ /** Whether to colour `stream`, from `env` (both read now, never at import). */
23
+ export function colorOn(stream, env) {
24
+ const e = env || {};
25
+ if (e.FORCE_COLOR !== undefined && e.FORCE_COLOR !== '') return e.FORCE_COLOR !== '0' && e.FORCE_COLOR !== 'false';
26
+ if (e.NO_COLOR !== undefined && e.NO_COLOR !== '') return false;
27
+ if (e.TERM === 'dumb') return false;
28
+ return !!(stream && stream.isTTY === true);
29
+ }
30
+
31
+ /** The colour helpers (identity functions when `on` is false). */
32
+ export function palette(on) {
33
+ if (!on) return { on: false, bold: plain, dim: plain, green: plain, yellow: plain, red: plain, cyan: plain };
34
+ return { on: true, bold: wrap(1, 22), dim: wrap(2, 22), green: wrap(32, 39), yellow: wrap(33, 39), red: wrap(31, 39), cyan: wrap(36, 39) };
35
+ }
36
+
37
+ /** Remove ANSI colour codes (for tests and for widths). */
38
+ export const stripAnsi = (s) => String(s).replace(/\u001b\[[0-9;]*m/g, '');
39
+
40
+ // ── elapsed time ─────────────────────────────────────────────────────────────────────────────────────────────
41
+ /** Milliseconds since `t0` (`now` defaults to Date.now()). */
42
+ export const elapsed = (t0, now) => Math.max(0, Math.round((now === undefined ? Date.now() : now) - t0));
43
+ /** `640 ms` — the unit every line uses, as the STUDIO terminal prints it. */
44
+ export const fmtMs = (ms) => (typeof ms === 'number' && Number.isFinite(ms) ? `${Math.round(ms)} ms` : '— ms');
45
+ /** `42 s ago`, `3 min ago`, `2 h ago`, `1 d ago` — the age of a cached step. */
46
+ export function fmtAge(ms) {
47
+ const s = Math.max(0, Math.floor(ms / 1000));
48
+ if (s < 5) return 'just now';
49
+ if (s < 60) return `${s} s ago`;
50
+ if (s < 3600) return `${Math.floor(s / 60)} min ago`;
51
+ if (s < 86400) return `${Math.floor(s / 3600)} h ago`;
52
+ return `${Math.floor(s / 86400)} d ago`;
53
+ }
54
+
55
+ // ── lines (pure: they return a string without a newline) ─────────────────────────────────────────────────────
56
+ const P0 = palette(false);
57
+ const tone = (mark, p) => (mark === 'ok' ? p.green : mark === 'warn' ? p.yellow : mark === 'fail' ? p.red : p.dim);
58
+
59
+ export const stepLine = (n, of, title, p = P0) => p.bold(`→ [${n}/${of}] ${title}`);
60
+ export const headLine = (text, p = P0) => p.bold(`→ ${text}`);
61
+ export function callLine(method, url, body, p = P0) {
62
+ return `● ${p.bold(method)} ${url}${body !== undefined ? ' ' + p.dim(JSON.stringify(body)) : ''}`;
63
+ }
64
+ export function calledLine(status, ms, error, p = P0) {
65
+ const t = `⎿ ${status === null || status === undefined ? 'no answer' : status} · ${fmtMs(ms)}${error ? ` · ${error}` : ''}`;
66
+ return ' ' + (error ? p.red(t) : p.dim(t));
67
+ }
68
+ /** `✓ text · 640 ms · extra` — `mark` is ok | warn | fail | skip. */
69
+ export function markLine(mark, text, o = {}, p = P0) {
70
+ const g = GLYPH[mark] || GLYPH.skip;
71
+ const tail = [o.ms !== undefined ? fmtMs(o.ms) : null, o.extra || null].filter(Boolean).join(' · ');
72
+ const body = mark === 'fail' ? p.red(text) : mark === 'skip' ? p.dim(text) : text;
73
+ return `${p.bold(tone(mark, p)(g))} ${body}${tail ? p.dim(` · ${tail}`) : ''}`;
74
+ }
75
+ export const noteLine = (text, p = P0) => ` ${p.dim(text)}`;
76
+ export const sayLine = (text) => ` ${text}`;
77
+ export const nextLine = (label, p = P0) => `${p.cyan('→')} ${label}`;
78
+ export const cmdLine = (cmd, p = P0) => String(cmd).split('\n').map((l) => ` ${p.dim('$')} ${l}`).join('\n');
79
+
80
+ // ── the writer ───────────────────────────────────────────────────────────────────────────────────────────────
81
+ /**
82
+ * A terminal bound to a stream. Options (all optional; defaults are read when this is called):
83
+ * out a writable with .write (default process.stdout)
84
+ * env the environment for NO_COLOR / FORCE_COLOR / TERM (default process.env)
85
+ * color force colour on or off
86
+ * quiet write nothing (for --json runs); the line functions still return their text
87
+ */
88
+ export function createTerm(o = {}) {
89
+ const proc = typeof process !== 'undefined' ? process : null;
90
+ const out = o.out || (proc && proc.stdout) || { write() {} };
91
+ const env = o.env || (proc && proc.env) || {};
92
+ const tty = !!out.isTTY;
93
+ const p = palette(o.color === undefined ? colorOn(out, env) : !!o.color);
94
+ const quiet = !!o.quiet;
95
+ let live = null; // the running spinner, cleared before any other line
96
+ const emit = (s) => {
97
+ if (quiet) return s;
98
+ if (live) live.clear();
99
+ out.write(s + '\n');
100
+ if (live) live.draw();
101
+ return s;
102
+ };
103
+ const t = {
104
+ color: p.on, tty, palette: p,
105
+ line: (s = '') => emit(s),
106
+ step: (n, of, title) => emit(stepLine(n, of, title, p)),
107
+ head: (text) => emit(headLine(text, p)),
108
+ call: (method, url, body) => emit(callLine(method, url, body, p)),
109
+ called: (status, ms, error) => emit(calledLine(status, ms, error, p)),
110
+ mark: (mark, text, x) => emit(markLine(mark, text, x, p)),
111
+ ok: (text, x) => emit(markLine('ok', text, x, p)),
112
+ warn: (text, x) => emit(markLine('warn', text, x, p)),
113
+ fail: (text, x) => emit(markLine('fail', text, x, p)),
114
+ skip: (text, x) => emit(markLine('skip', text, x, p)),
115
+ note: (text) => emit(noteLine(text, p)),
116
+ say: (text) => emit(sayLine(text)),
117
+ next: (label) => emit(nextLine(label, p)),
118
+ cmd: (cmd) => emit(cmdLine(cmd, p)),
119
+ info: (text) => emit(p.dim(text)),
120
+ /** one JSON value per line, written even when quiet (that is what --json prints) */
121
+ json: (v) => { out.write(JSON.stringify(v) + '\n'); return v; },
122
+ /** A spinner on its own line; animates only on a colour TTY. stop(line?) clears it and prints `line` if given. */
123
+ spinner(text) {
124
+ if (quiet || !tty || !p.on) return { update() {}, stop: (line) => (line ? emit(line) : undefined) };
125
+ let f = 0; let label = text;
126
+ const s = {
127
+ draw: () => out.write(`${p.dim(SPIN[f % SPIN.length])} ${label}`),
128
+ clear: () => out.write('\r\u001b[2K'),
129
+ };
130
+ live = s; s.draw();
131
+ const timer = setInterval(() => { f++; s.clear(); s.draw(); }, 90);
132
+ if (timer && typeof timer.unref === 'function') timer.unref();
133
+ return {
134
+ update(next) { label = next; },
135
+ stop(line) { clearInterval(timer); s.clear(); if (live === s) live = null; return line ? emit(line) : undefined; },
136
+ };
137
+ },
138
+ };
139
+ return t;
140
+ }
141
+
142
+ // ── the default terminal: process.stdout, created on first use (never at import) ─────────────────────────────
143
+ let dflt = null;
144
+ const d = () => (dflt || (dflt = createTerm()));
145
+ export const step = (n, of, title) => d().step(n, of, title);
146
+ export const call = (method, url, body) => d().call(method, url, body);
147
+ export const called = (status, ms, error) => d().called(status, ms, error);
148
+ export const ok = (text, x) => d().ok(text, x);
149
+ export const warn = (text, x) => d().warn(text, x);
150
+ export const fail = (text, x) => d().fail(text, x);
151
+ export const skip = (text, x) => d().skip(text, x);
152
+ export const note = (text) => d().note(text);
153
+ export const next = (label) => d().next(label);
154
+ export const spinner = (text) => d().spinner(text);