@taifoon/jev 0.2.6 → 0.2.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/docs/WORKFLOWS.md +9 -7
- package/package.json +1 -1
- package/workflows/0.2.7/README.md +107 -0
- package/workflows/0.2.7/batch-judge.json +341 -0
- package/workflows/0.2.7/hire-judge-settle.json +819 -0
- package/workflows/0.2.7/jev-grader.json +433 -0
- package/workflows/0.2.7/manifest.json +151 -0
- package/workflows/0.2.7/schemas/answers.json +230 -0
- package/workflows/0.2.7/schemas/common.json +394 -0
- package/workflows/0.2.7/schemas/delivery.json +217 -0
- package/workflows/0.2.7/schemas/facts.json +209 -0
- package/workflows/0.2.7/schemas/hire.json +314 -0
- package/workflows/0.2.7/schemas/job.json +88 -0
- package/workflows/0.2.7/schemas/settlement.json +209 -0
- package/workflows/0.2.7/schemas/verdict.json +296 -0
- package/workflows/0.2.7/stamp-grade.json +175 -0
package/README.md
CHANGED
|
@@ -91,7 +91,7 @@ npx skills add taifoon-io/jev --skill jev-grader
|
|
|
91
91
|
## In n8n
|
|
92
92
|
|
|
93
93
|
The same grader runs in n8n through [`@taifoon/n8n-nodes-typesafe`](https://www.npmjs.com/package/@taifoon/n8n-nodes-typesafe),
|
|
94
|
-
with
|
|
94
|
+
with twelve ready-to-import judge workflows: grade an agent's delivery, grade a Base job from its record, fact-check a
|
|
95
95
|
chatbot answer, settle a refund dispute and more. See [its README](https://github.com/taifoon-io/n8n-nodes-typesafe#judge-workflows-ready-to-import).
|
|
96
96
|
|
|
97
97
|
## `jev run` flags
|
package/docs/WORKFLOWS.md
CHANGED
|
@@ -13,22 +13,24 @@ The node outputs the receipt and the unsigned calls, and it signs nothing. With
|
|
|
13
13
|
|
|
14
14
|
## Workflows
|
|
15
15
|
|
|
16
|
-
`workflows/0.2.
|
|
17
|
-
GitHub release as `jev-workflows-0.2.
|
|
16
|
+
`workflows/0.2.7/` holds the four n8n workflows that run Jev (`workflows/0.2.2/` is kept as it shipped). The same
|
|
17
|
+
files are attached to the GitHub release as `jev-workflows-0.2.7.zip`:
|
|
18
18
|
|
|
19
19
|
- **`hire-judge-settle.json`** runs hire → prepare (facts) → TypeSafe (the four RUBRIC_v1 questions) → answers
|
|
20
|
-
(verdict, recorded
|
|
20
|
+
(verdict, recorded; Record On = devnet anchors it) → settle on devnet. Executions 86 and 88 of this workflow are the
|
|
21
|
+
golden runs above. Needs a Taifoon relayer key, your TypeSafe key and two devnet keys; outputs the whole job with every
|
|
22
|
+
transaction and its proof link.
|
|
21
23
|
- **`jev-grader.json`** grades Base jobs nobody ruled on, then stamps each verdict on the devnet GradeStampRegistry.
|
|
22
24
|
- **`batch-judge.json`** runs prepare → Jev → answers for each subject, then `POST /v1/judge/batch`.
|
|
23
25
|
- **`stamp-grade.json`** takes a recorded decision, turns it into the canonical verdict and stamps it.
|
|
24
26
|
|
|
25
27
|
Credential references are placeholders: `{ "id": "", "name": "REPLACE: <credential type>" }`. After import, pick your
|
|
26
28
|
own credential of that type. Each step names the entity it takes and emits by `$id` (`meta.taifoon.steps`). The schemas
|
|
27
|
-
behind those `$id`s are in `workflows/0.2.
|
|
28
|
-
`workflows/0.2.
|
|
29
|
+
behind those `$id`s are in `workflows/0.2.7/schemas/`. `manifest.json` pins every file by sha256.
|
|
30
|
+
`workflows/0.2.7/README.md` explains each workflow step by step and lists the nodes and credentials it needs:
|
|
29
31
|
|
|
30
|
-
- `@taifoon/n8n-nodes-typesafe`, tested with
|
|
31
|
-
- `n8n-nodes-taifoon`, tested with 0.4.
|
|
32
|
+
- `@taifoon/n8n-nodes-typesafe`, tested with 2.0.1, is on npm.
|
|
33
|
+
- `n8n-nodes-taifoon`, tested with 0.4.5, and `n8n-nodes-taifoon-devnet-signer`, tested with 0.3.0, are custom
|
|
32
34
|
extensions and are not on npm.
|
|
33
35
|
|
|
34
36
|
```
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@taifoon/jev",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.7",
|
|
4
4
|
"description": "Grade AI agent jobs with TypeSafe's Jev and record the verdict on chain: code checks the facts, Jev answers a published rubric, and you get a receipt, the on-chain record and the evaluator call for ERC-8183 and Virtuals ACP. No runtime dependencies, no key inside.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"jev",
|
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
# Jev workflows 0.2.7
|
|
2
|
+
|
|
3
|
+
The n8n workflows that run Jev on n8n.taifoon.dev, exported with credential references replaced by placeholders (`{ "id": "", "name": "REPLACE: <credential type>" }`: pick your own credential of that type after import). Import with n8n → Workflows → Import from File, or `n8n import:workflow --input=<file>`. Every step carries its contract: `meta.taifoon.steps` names the canonical entity it takes and emits by `$id`; the schemas are in `schemas/` (the `$id`s are identifiers, the files are here).
|
|
4
|
+
|
|
5
|
+
## Nodes they need
|
|
6
|
+
|
|
7
|
+
| Package | Tested with | Where |
|
|
8
|
+
|---|---|---|
|
|
9
|
+
| `@taifoon/n8n-nodes-typesafe` | 2.0.1 | npm (community node): npm i @taifoon/n8n-nodes-typesafe |
|
|
10
|
+
| `n8n-nodes-taifoon` | 0.4.5 | custom extension (type prefix CUSTOM.); not on npm |
|
|
11
|
+
| `n8n-nodes-taifoon-devnet-signer` | 0.3.0 | custom extension (type prefix CUSTOM.); not on npm |
|
|
12
|
+
|
|
13
|
+
## Credentials they need
|
|
14
|
+
|
|
15
|
+
| Type | What |
|
|
16
|
+
|---|---|
|
|
17
|
+
| `taifoonTypeSafeApi` | your TypeSafe key (console.typesafe.ai) — credential type "TypeSafe API" of @taifoon/n8n-nodes-typesafe |
|
|
18
|
+
| `taifoonRelayerApi` | a Taifoon relayer key (tfr_…) for https://www.taifoon.io/v1 — credential type "Taifoon Relayer API" of n8n-nodes-taifoon |
|
|
19
|
+
| `taifoonDevnetKeyApi` | a devnet 36927 key with gas from https://faucet.taifoon.dev — credential type "Taifoon Devnet Key API" of n8n-nodes-taifoon-devnet-signer |
|
|
20
|
+
|
|
21
|
+
## HIRE → JUDGE → SETTLE (devnet 36927) — `hire-judge-settle.json`
|
|
22
|
+
|
|
23
|
+
One hire from offer to settlement. The seller’s reply is held with its digest; prepare decides the facts (a hard fail skips Jev); your TypeSafe credential answers the four RUBRIC_v1 questions; answers composes the verdict under THRESHOLDS_v1, records decision.v2 and jev.answer.v1 and, with Record On = devnet (set in this file), anchors them on the devnet logs; a complete settles through the devnet assurance hook (approve, fund, submit, complete), needs_review is held until the job’s deadline, when anyone may call expire. Executions 86 and 88 of this workflow are the golden runs @taifoon/jev re-derives.
|
|
24
|
+
|
|
25
|
+
**Needs:** n8n-nodes-taifoon ≥ 0.4.5 and n8n-nodes-taifoon-devnet-signer ≥ 0.3.0 (custom extensions), @taifoon/n8n-nodes-typesafe ≥ 2.0.1; a Taifoon relayer key, your TypeSafe key, and two devnet 36927 keys (buyer, seller) with gas and dUSDC. **Outputs:** the whole JOB (Receipt, then Proofs · wait): handshake, verdict, decision with its devnet anchor tx, and every settlement transaction with its proof link.
|
|
26
|
+
|
|
27
|
+
- source: `tfnhiresettle001` on n8n.taifoon.dev · sha256 `2f746f9cd8c46b0342d99eb18b555696b9cfbfa3118b83905d7f92453e53da06`
|
|
28
|
+
- nodes: `n8n-nodes-taifoon`, `@taifoon/n8n-nodes-typesafe`, `n8n-nodes-taifoon-devnet-signer`; built-ins manualTrigger v1, set v3.4, code v2, if v2.2
|
|
29
|
+
- credentials: `taifoonRelayerApi`, `taifoonTypeSafeApi`, `taifoonDevnetKeyApi`
|
|
30
|
+
|
|
31
|
+
| Step | Takes | Emits | Does |
|
|
32
|
+
|---|---|---|---|
|
|
33
|
+
| Start | — | — | start a run (swap for a Webhook or Schedule trigger) |
|
|
34
|
+
| Inputs | — | hire.json | the hire intent: task, candidate seller, endpoint/tool, buyer, price, chain (a HIRE in state Open) |
|
|
35
|
+
| Offer + deliver | hire.json | job.json | POST /v1/handshake {dispatch:true}: the offer goes to the seller’s own endpoint; the reply is held with its keccak digest |
|
|
36
|
+
| Delivery | job.json | delivery.json | stop unless a reply digest is held (no digest = nothing to settle) |
|
|
37
|
+
| Judge · prepare | delivery.json | facts.json | POST /v1/judge/compose (prepare): the facts decide first; hard_fail ends here; else the exact text Jev reads + prepare_digest |
|
|
38
|
+
| Hard fail? | facts.json | facts.json | branch: a hard fail goes straight to the verdict without asking Jev |
|
|
39
|
+
| Jev · TypeSafe (our credential) | facts.json | answers.json | TypeSafe ask on the prepared text: every RUBRIC_ASKED answer with its whole distribution |
|
|
40
|
+
| Judge · answers | answers.json | verdict.json | POST /v1/judge/compose (answers): recompose under THRESHOLDS_v1, record decision.v2, anchor on JevDecisionLog |
|
|
41
|
+
| Verdict | verdict.json + facts.json | verdict.json | the composed verdict (complete \| reject \| needs_review) with what settlement needs carried forward |
|
|
42
|
+
| Settle | verdict.json | settlement.json | POST /v1/settle: the plan (jobId, price, premium, deposit, calls[] in order, who signs each) |
|
|
43
|
+
| Seller · approve deposit | settlement.json | settlement.json | the seller approves D (> P); the tx joins settlement.txs |
|
|
44
|
+
| Buyer · approve + fund | settlement.json | settlement.json | the buyer approves P + π and calls fundJob: machine Delivered → Funded |
|
|
45
|
+
| Seller · submit | settlement.json | settlement.json | submit(jobId, evidenceDigest = delivery.digest): machine Funded → Submitted |
|
|
46
|
+
| Verdict says complete? | settlement.json | settlement.json | branch on the composed verdict: complete → complete; otherwise hold (Submitted → Held) |
|
|
47
|
+
| Buyer · complete + capture | settlement.json | settlement.json | complete(jobId) (+ capture): machine → Settled |
|
|
48
|
+
| Held for review | settlement.json | settlement.json | needs_review / reject: funded and sealed, nothing paid; the evaluator leg or the deadline decides |
|
|
49
|
+
| Receipt | settlement.json | job.json | the whole JOB: hire, delivery, facts, verdict, settlement — every transaction with its proof link |
|
|
50
|
+
| Proofs · wait | job.json | job.json | GET /v1/proof/tx/36927/<tx> for every transaction of the run; pending is retried with backoff (5 s -> 60 s, <= 240 s), never reported as a gap |
|
|
51
|
+
|
|
52
|
+
## Jev on-chain grader (grade → stamp → devnet) — `jev-grader.json`
|
|
53
|
+
|
|
54
|
+
Grades Base jobs that were paid and delivered and that nobody ruled on (GET /v1/judge/ready): four per run, each through prepare → Jev → answers, then the canonical verdict, then GradeStampRegistry.stamp on the devnet, signed by the devnet signer node; the stamp transaction is appended to the verdict’s anchors.
|
|
55
|
+
|
|
56
|
+
- source: `tfnjevgrader0001` on n8n.taifoon.dev · sha256 `762248c554dd0607362e20361ec3efe0fda58326d89c2f1c18c525ea48e92e35`
|
|
57
|
+
- nodes: `n8n-nodes-taifoon`, `@taifoon/n8n-nodes-typesafe`, `n8n-nodes-taifoon-devnet-signer`; built-ins manualTrigger v1, code v2, if v2.2
|
|
58
|
+
- credentials: `taifoonRelayerApi`, `taifoonTypeSafeApi`, `taifoonDevnetKeyApi`
|
|
59
|
+
|
|
60
|
+
| Step | Takes | Emits | Does |
|
|
61
|
+
|---|---|---|---|
|
|
62
|
+
| When clicking ‘Execute workflow’ | — | — | start a grading run |
|
|
63
|
+
| Ready jobs (Base) | — | job.json | GET /v1/judge/ready: jobs on Base that were paid and delivered, and nobody ruled |
|
|
64
|
+
| Pick 4 calibrated | job.json | job.json | keep four whose seller has a calibrated record; one item per job, no state built here |
|
|
65
|
+
| Judge · prepare | job.json | facts.json | POST /v1/judge/compose (prepare) on the job: facts decide first; the pack comes from the job evidence (GRADEABLE labels 1–21) |
|
|
66
|
+
| Hard fail? | facts.json | facts.json | a hard fail is final at prepare (Jev is not asked; nothing to stamp) |
|
|
67
|
+
| Jev · TypeSafe (our credential) | facts.json | answers.json | TypeSafe ask on the prepared text (jevStateOf): every RUBRIC_ASKED answer with its whole distribution |
|
|
68
|
+
| Judge · answers | answers.json | verdict.json | POST /v1/judge/compose (answers): recompose under THRESHOLDS_v1, record decision.v2 and the answers, anchor both |
|
|
69
|
+
| Verdict | verdict.json | verdict.json | the recorded decision as a canonical VERDICT (fromDecision), composed from this run’s receipt |
|
|
70
|
+
| Stamp Grade (unsigned) | verdict.json | verdict.json | POST /v1/assurance/call stampGrade: the unsigned GradeStampRegistry.stamp(subject_id, digest, verdict code, mode 2) |
|
|
71
|
+
| Taifoon Devnet Signer | verdict.json | verdict.json | sign and send the stamp on devnet 36927 (anvil #0) |
|
|
72
|
+
| Verdict + stamp | verdict.json | verdict.json | the same VERDICT with the stamp transaction appended to anchors |
|
|
73
|
+
|
|
74
|
+
## Batch judge — `batch-judge.json`
|
|
75
|
+
|
|
76
|
+
A batch of subjects, each as its own prepare → Jev → answers (so every answer is recorded), aggregated and posted to POST /v1/judge/batch.
|
|
77
|
+
|
|
78
|
+
- source: `tfnbatchjudge001` on n8n.taifoon.dev · sha256 `8e66a8e261e9f1f8052bd0b7658cb2ef5212d8df89b8a5347b56762701f078fd`
|
|
79
|
+
- nodes: `n8n-nodes-taifoon`, `@taifoon/n8n-nodes-typesafe`; built-ins manualTrigger v1, code v2, if v2.2, httpRequest v4.2
|
|
80
|
+
- credentials: `taifoonRelayerApi`, `taifoonTypeSafeApi`
|
|
81
|
+
|
|
82
|
+
| Step | Takes | Emits | Does |
|
|
83
|
+
|---|---|---|---|
|
|
84
|
+
| Grade a batch | — | — | start a batch (pass { subjects, jobId, threshold } or run the demo set) |
|
|
85
|
+
| Build the batch | — | hire.json | one canonical HIRE per subject (a delivered handshake, or a job on chain); no state is built here |
|
|
86
|
+
| Judge · prepare | hire.json | facts.json | POST /v1/judge/compose (prepare) per subject: the facts decide first; the pack is built from DELIVERY + FACTS; else the exact text Jev reads + prepare_digest |
|
|
87
|
+
| Hard fail? | facts.json | facts.json | branch: a hard fail is final (Jev is not asked) and goes straight to the batch |
|
|
88
|
+
| Jev · TypeSafe (our credential) | facts.json | answers.json | TypeSafe ask on the prepared text (jevStateOf): every RUBRIC_ASKED answer with its whole distribution |
|
|
89
|
+
| Judge · answers | answers.json | verdict.json | POST /v1/judge/compose (answers): recompose under THRESHOLDS_v1, record decision.v2 and the answers (jev.answer.v1), anchor both |
|
|
90
|
+
| Grade + aggregate | facts.json + verdict.json | — | the batch outcome from the recorded verdicts (Conformant / NonConformant / NoGrade counts, fail fraction vs threshold) — a batch record, not a canonical entity |
|
|
91
|
+
| Record the batch to the coordination layer | — | — | POST /v1/judge/batch (relayer key): the batch record, keyed batch:<jobId> |
|
|
92
|
+
|
|
93
|
+
## Stamp Grade (a recorded verdict → GradeStampRegistry) — `stamp-grade.json`
|
|
94
|
+
|
|
95
|
+
Takes one recorded decision (by id), maps it to the canonical verdict, and stamps it on the devnet GradeStampRegistry; emits the verdict with the stamp transaction in its anchors.
|
|
96
|
+
|
|
97
|
+
- source: `tfnstamp00000001` on n8n.taifoon.dev · sha256 `516d9483e3d26e1891dacb8c4493d609340e3c6a3b200784c72864f9cb931ebc`
|
|
98
|
+
- nodes: `n8n-nodes-taifoon`, `n8n-nodes-taifoon-devnet-signer`; built-ins manualTrigger v1, code v2
|
|
99
|
+
- credentials: `taifoonDevnetKeyApi`
|
|
100
|
+
|
|
101
|
+
| Step | Takes | Emits | Does |
|
|
102
|
+
|---|---|---|---|
|
|
103
|
+
| Start | — | — | start (pass { decision_id } to stamp another verdict) |
|
|
104
|
+
| Verdict | — | verdict.json | GET /v1/judge/decisions/:id → the canonical VERDICT (fromDecision) and its stamp code |
|
|
105
|
+
| Stamp Grade | verdict.json | verdict.json | POST /v1/assurance/call stampGrade: the unsigned GradeStampRegistry.stamp(subject_id, digest, verdict code, mode) |
|
|
106
|
+
| Taifoon Devnet Signer | verdict.json | verdict.json | sign and send the stamp on devnet 36927 (anvil #0) |
|
|
107
|
+
| Verdict + stamp | verdict.json | verdict.json | the same VERDICT with the stamp transaction appended to anchors |
|
|
@@ -0,0 +1,341 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "Taifoon — batch judge scores for the coordination layer (TypeSafe/Jev)",
|
|
3
|
+
"nodes": [
|
|
4
|
+
{
|
|
5
|
+
"parameters": {},
|
|
6
|
+
"id": "trigger",
|
|
7
|
+
"name": "Grade a batch",
|
|
8
|
+
"type": "n8n-nodes-base.manualTrigger",
|
|
9
|
+
"typeVersion": 1,
|
|
10
|
+
"position": [
|
|
11
|
+
200,
|
|
12
|
+
300
|
|
13
|
+
],
|
|
14
|
+
"notes": "in: — → out: — · start a batch (pass { subjects, jobId, threshold } or run the demo set)",
|
|
15
|
+
"notesInFlow": true
|
|
16
|
+
},
|
|
17
|
+
{
|
|
18
|
+
"parameters": {
|
|
19
|
+
"jsCode": "// The batch: subjects already delivered (a handshake, or a job read from the chain), one canonical HIRE each. The\n// judge builds every evidence pack itself from DELIVERY + FACTS (schemas/GRADEABLE.md) — nothing here hands Jev a\n// state. Pass { subjects: [{ handshake_id } | { chainId, jobId }], jobId, threshold } on the trigger, or run the demo set.\nconst t = $input.first().json || {};\nconst subjects = Array.isArray(t.subjects) && t.subjects.length ? t.subjects : [\n { handshake_id: 'hs_97dcf5942e294f7ca28efd0a' }, { handshake_id: 'hs_ec150f74dbe6568982c6c93d' }, { handshake_id: 'hs_c07ccb807a29b94dffb18459' },\n];\nreturn subjects.map((s) => {\n const hs = typeof s.handshake_id === 'string' && /^hs_[0-9a-f]{24}$/.test(s.handshake_id) ? s.handshake_id : null;\n const chainId = s.chainId != null ? Number(s.chainId) : hs ? 36927 : 8453;\n const id = hs ?? String(s.jobId);\n const job = hs ? { key: 'hs:' + hs, lane: 'handshake', chain_id: chainId, id: hs, contract: null }\n : { key: 'erc8183:' + chainId + ':' + id, lane: 'erc8183', chain_id: chainId, id, contract: '0x238e541bfefd82238730d00a2208e5497f1832e0' };\n return { json: {\n schema: 'taifoon.hire.v1', job, lane: job.lane, tenant: null, buyer: null, seller: null, seller_agent_key: null, seller_ids: [],\n evaluator: null, hook: null, evaluator_kind: null, task: null, required_skills: [], budget: null, funding: null,\n created_at: null, deadline: null, handshake_id: hs, offer_nonce: null, state: 'Delivered', ending: null,\n anchors: [{ kind: 'offchain', store: hs ? 'coordd:coordination_handshakes' : 'observatory', key: id, digest: null, anchored_by: null, what: 'the subject this batch grades' }],\n source: { kind: hs ? 'handshake' : 'observatory-job', url: hs ? 'https://www.taifoon.io/v1/handshake/' + hs : 'https://www.taifoon.io/v1/judge/evidence/' + chainId + '/' + id, read_at: Math.floor(Date.now() / 1000) },\n } };\n});"
|
|
20
|
+
},
|
|
21
|
+
"id": "e177aad3-f294-4ac5-beae-395993bd7de5",
|
|
22
|
+
"name": "Build the batch",
|
|
23
|
+
"type": "n8n-nodes-base.code",
|
|
24
|
+
"typeVersion": 2,
|
|
25
|
+
"position": [
|
|
26
|
+
420,
|
|
27
|
+
300
|
|
28
|
+
],
|
|
29
|
+
"notes": "in: — → out: hire.json · one canonical HIRE per subject (a delivered handshake, or a job on chain); no state is built here",
|
|
30
|
+
"notesInFlow": true
|
|
31
|
+
},
|
|
32
|
+
{
|
|
33
|
+
"name": "Judge · prepare",
|
|
34
|
+
"type": "CUSTOM.taifoon",
|
|
35
|
+
"typeVersion": 1,
|
|
36
|
+
"position": [
|
|
37
|
+
640,
|
|
38
|
+
300
|
|
39
|
+
],
|
|
40
|
+
"parameters": {
|
|
41
|
+
"resource": "judge",
|
|
42
|
+
"operation": "compose",
|
|
43
|
+
"composeSubject": "={{ $json.lane === 'handshake' ? 'handshake' : 'job' }}",
|
|
44
|
+
"composeHandshakeId": "={{ $json.handshake_id }}",
|
|
45
|
+
"composeMode": "prepare",
|
|
46
|
+
"composeJobId": "={{ $json.job.id }}",
|
|
47
|
+
"composeChainId": "={{ $json.job.chain_id }}"
|
|
48
|
+
},
|
|
49
|
+
"credentials": {
|
|
50
|
+
"taifoonRelayerApi": {
|
|
51
|
+
"id": "",
|
|
52
|
+
"name": "REPLACE: taifoonRelayerApi"
|
|
53
|
+
}
|
|
54
|
+
},
|
|
55
|
+
"id": "62dca0ae-d853-414f-ac98-89592623c7d2",
|
|
56
|
+
"notes": "in: hire.json → out: facts.json · POST /v1/judge/compose (prepare) per subject: the facts decide first; the pack is built from DELIVERY + FACTS; else the exact text Jev reads + prepare_digest",
|
|
57
|
+
"notesInFlow": true
|
|
58
|
+
},
|
|
59
|
+
{
|
|
60
|
+
"name": "Hard fail?",
|
|
61
|
+
"type": "n8n-nodes-base.if",
|
|
62
|
+
"typeVersion": 2.2,
|
|
63
|
+
"position": [
|
|
64
|
+
860,
|
|
65
|
+
300
|
|
66
|
+
],
|
|
67
|
+
"parameters": {
|
|
68
|
+
"conditions": {
|
|
69
|
+
"options": {
|
|
70
|
+
"caseSensitive": true,
|
|
71
|
+
"leftValue": "",
|
|
72
|
+
"typeValidation": "strict",
|
|
73
|
+
"version": 2
|
|
74
|
+
},
|
|
75
|
+
"conditions": [
|
|
76
|
+
{
|
|
77
|
+
"id": "h1",
|
|
78
|
+
"leftValue": "={{ $json.hard_fail === true }}",
|
|
79
|
+
"rightValue": "",
|
|
80
|
+
"operator": {
|
|
81
|
+
"type": "boolean",
|
|
82
|
+
"operation": "true",
|
|
83
|
+
"singleValue": true
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
],
|
|
87
|
+
"combinator": "and"
|
|
88
|
+
},
|
|
89
|
+
"options": {}
|
|
90
|
+
},
|
|
91
|
+
"id": "65ffd12f-ea5f-4a36-98c4-eea5f9afec86",
|
|
92
|
+
"notes": "in: facts.json → out: facts.json · branch: a hard fail is final (Jev is not asked) and goes straight to the batch",
|
|
93
|
+
"notesInFlow": true
|
|
94
|
+
},
|
|
95
|
+
{
|
|
96
|
+
"name": "Jev · TypeSafe (our credential)",
|
|
97
|
+
"type": "@taifoon/n8n-nodes-typesafe.taifoonTypeSafe",
|
|
98
|
+
"typeVersion": 1,
|
|
99
|
+
"position": [
|
|
100
|
+
1080,
|
|
101
|
+
420
|
|
102
|
+
],
|
|
103
|
+
"parameters": {
|
|
104
|
+
"operation": "ask",
|
|
105
|
+
"connection": "direct",
|
|
106
|
+
"state": "={{ JSON.stringify($json.jev.state) }}",
|
|
107
|
+
"questions": {},
|
|
108
|
+
"questionsJson": "={{ JSON.stringify($json.jev.questions) }}",
|
|
109
|
+
"routing": "{}",
|
|
110
|
+
"replyLanguage": "off",
|
|
111
|
+
"failClosed": true
|
|
112
|
+
},
|
|
113
|
+
"credentials": {
|
|
114
|
+
"taifoonTypeSafeApi": {
|
|
115
|
+
"id": "",
|
|
116
|
+
"name": "REPLACE: taifoonTypeSafeApi"
|
|
117
|
+
}
|
|
118
|
+
},
|
|
119
|
+
"notes": "in: facts.json → out: answers.json · TypeSafe ask on the prepared text (jevStateOf): every RUBRIC_ASKED answer with its whole distribution",
|
|
120
|
+
"id": "f2965735-6523-490f-a551-865edd654baf",
|
|
121
|
+
"notesInFlow": true
|
|
122
|
+
},
|
|
123
|
+
{
|
|
124
|
+
"name": "Judge · answers",
|
|
125
|
+
"type": "CUSTOM.taifoon",
|
|
126
|
+
"typeVersion": 1,
|
|
127
|
+
"position": [
|
|
128
|
+
1300,
|
|
129
|
+
420
|
|
130
|
+
],
|
|
131
|
+
"parameters": {
|
|
132
|
+
"resource": "judge",
|
|
133
|
+
"operation": "compose",
|
|
134
|
+
"composeSubject": "={{ $('Build the batch').item.json.lane === 'handshake' ? 'handshake' : 'job' }}",
|
|
135
|
+
"composeHandshakeId": "={{ $('Build the batch').item.json.handshake_id }}",
|
|
136
|
+
"composeMode": "answers",
|
|
137
|
+
"composeAnswers": "={{ JSON.stringify($json.answers) }}",
|
|
138
|
+
"composeModel": "={{ $json.model }}",
|
|
139
|
+
"composePrepareDigest": "={{ $('Judge · prepare').item.json.prepare_digest }}",
|
|
140
|
+
"composeAnsweredBy": "n8n-typesafe",
|
|
141
|
+
"composeExecutionId": "={{ $execution.id }}",
|
|
142
|
+
"composeLatencyMs": "={{ $json.latency_ms }}",
|
|
143
|
+
"composeJobId": "={{ $('Build the batch').item.json.job.id }}",
|
|
144
|
+
"composeChainId": "={{ $('Build the batch').item.json.job.chain_id }}"
|
|
145
|
+
},
|
|
146
|
+
"credentials": {
|
|
147
|
+
"taifoonRelayerApi": {
|
|
148
|
+
"id": "",
|
|
149
|
+
"name": "REPLACE: taifoonRelayerApi"
|
|
150
|
+
}
|
|
151
|
+
},
|
|
152
|
+
"id": "0ca9f21a-aff6-4fa1-8cc7-c0924dc1cce3",
|
|
153
|
+
"notes": "in: answers.json → out: verdict.json · POST /v1/judge/compose (answers): recompose under THRESHOLDS_v1, record decision.v2 and the answers (jev.answer.v1), anchor both",
|
|
154
|
+
"notesInFlow": true
|
|
155
|
+
},
|
|
156
|
+
{
|
|
157
|
+
"parameters": {
|
|
158
|
+
"jsCode": "// The batch outcome from the recorded verdicts — each item was composed under THRESHOLDS_v1 and recorded (decision.v2\n// + jev.answer.v1) by Judge · answers; a hard fail was final at prepare. complete → Conformant, reject → NonConformant,\n// needs_review → NoGrade (the verdict crate's batch mapping). Runs once per branch that reaches it; the batch record\n// is keyed by jobId, so the last (complete) run is the one that stays.\nconst t = $('Grade a batch').first().json || {};\nconst jobId = t.jobId || 'batch-' + $execution.id;\nconst threshold = typeof t.threshold === 'number' ? t.threshold : 0.2;\nconst prepared = $('Judge · prepare').all().map((i) => i.json);\nlet answered = [];\ntry { answered = $('Judge · answers').all().map((i) => i.json); } catch (e) { answered = []; }\nconst items = [];\nfor (const p of prepared.filter((x) => x.hard_fail === true)) items.push({ item_id: p.subject, verdict: 'NonConformant', composed: 'reject', forced: 'hard_fail', mode: 'Deterministic', decision_id: null, answers_digest: null });\nfor (const a of answered) {\n const v = a.receipt && a.receipt.verdict;\n items.push({ item_id: (a.decision && a.decision.subject_id) || null, verdict: v === 'complete' ? 'Conformant' : v === 'reject' ? 'NonConformant' : 'NoGrade', composed: v || null, forced: null, mode: 'Calibrated',\n decision_id: a.decision ? a.decision.id : null, decision_digest: a.decision ? a.decision.digest : null, answers_digest: a.decision ? a.decision.answers_digest ?? null : null });\n}\nconst conformant = items.filter((i) => i.verdict === 'Conformant').length, nonconformant = items.filter((i) => i.verdict === 'NonConformant').length, nograde = items.length - conformant - nonconformant;\nconst graded = conformant + nonconformant, failFraction = graded === 0 ? 1 : nonconformant / graded;\nreturn [{ json: { jobId, mode: 'Calibrated', total: items.length, conformant, nonconformant, nograde, failFraction, threshold,\n verdict: graded === 0 ? 'NoGrade' : failFraction <= threshold ? 'Conformant' : 'NonConformant', items } }];"
|
|
159
|
+
},
|
|
160
|
+
"id": "64dce113-21e8-454e-8a12-53858bd7a303",
|
|
161
|
+
"name": "Grade + aggregate",
|
|
162
|
+
"type": "n8n-nodes-base.code",
|
|
163
|
+
"typeVersion": 2,
|
|
164
|
+
"position": [
|
|
165
|
+
1520,
|
|
166
|
+
300
|
|
167
|
+
],
|
|
168
|
+
"notes": "in: facts.json | verdict.json → out: — · the batch outcome from the recorded verdicts (Conformant / NonConformant / NoGrade counts, fail fraction vs threshold) — a batch record, not a canonical entity",
|
|
169
|
+
"notesInFlow": true
|
|
170
|
+
},
|
|
171
|
+
{
|
|
172
|
+
"parameters": {
|
|
173
|
+
"method": "POST",
|
|
174
|
+
"url": "https://www.taifoon.io/v1/judge/batch",
|
|
175
|
+
"authentication": "predefinedCredentialType",
|
|
176
|
+
"nodeCredentialType": "taifoonRelayerApi",
|
|
177
|
+
"sendBody": true,
|
|
178
|
+
"specifyBody": "json",
|
|
179
|
+
"jsonBody": "={{ JSON.stringify($json) }}",
|
|
180
|
+
"options": {}
|
|
181
|
+
},
|
|
182
|
+
"id": "record",
|
|
183
|
+
"name": "Record the batch to the coordination layer",
|
|
184
|
+
"type": "n8n-nodes-base.httpRequest",
|
|
185
|
+
"typeVersion": 4.2,
|
|
186
|
+
"position": [
|
|
187
|
+
1740,
|
|
188
|
+
300
|
|
189
|
+
],
|
|
190
|
+
"credentials": {
|
|
191
|
+
"taifoonRelayerApi": {
|
|
192
|
+
"id": "",
|
|
193
|
+
"name": "REPLACE: taifoonRelayerApi"
|
|
194
|
+
}
|
|
195
|
+
},
|
|
196
|
+
"notes": "in: — → out: — · POST /v1/judge/batch (relayer key): the batch record, keyed batch:<jobId>",
|
|
197
|
+
"notesInFlow": true
|
|
198
|
+
}
|
|
199
|
+
],
|
|
200
|
+
"connections": {
|
|
201
|
+
"Grade a batch": {
|
|
202
|
+
"main": [
|
|
203
|
+
[
|
|
204
|
+
{
|
|
205
|
+
"node": "Build the batch",
|
|
206
|
+
"type": "main",
|
|
207
|
+
"index": 0
|
|
208
|
+
}
|
|
209
|
+
]
|
|
210
|
+
]
|
|
211
|
+
},
|
|
212
|
+
"Build the batch": {
|
|
213
|
+
"main": [
|
|
214
|
+
[
|
|
215
|
+
{
|
|
216
|
+
"node": "Judge · prepare",
|
|
217
|
+
"type": "main",
|
|
218
|
+
"index": 0
|
|
219
|
+
}
|
|
220
|
+
]
|
|
221
|
+
]
|
|
222
|
+
},
|
|
223
|
+
"Judge · prepare": {
|
|
224
|
+
"main": [
|
|
225
|
+
[
|
|
226
|
+
{
|
|
227
|
+
"node": "Hard fail?",
|
|
228
|
+
"type": "main",
|
|
229
|
+
"index": 0
|
|
230
|
+
}
|
|
231
|
+
]
|
|
232
|
+
]
|
|
233
|
+
},
|
|
234
|
+
"Hard fail?": {
|
|
235
|
+
"main": [
|
|
236
|
+
[
|
|
237
|
+
{
|
|
238
|
+
"node": "Grade + aggregate",
|
|
239
|
+
"type": "main",
|
|
240
|
+
"index": 0
|
|
241
|
+
}
|
|
242
|
+
],
|
|
243
|
+
[
|
|
244
|
+
{
|
|
245
|
+
"node": "Jev · TypeSafe (our credential)",
|
|
246
|
+
"type": "main",
|
|
247
|
+
"index": 0
|
|
248
|
+
}
|
|
249
|
+
]
|
|
250
|
+
]
|
|
251
|
+
},
|
|
252
|
+
"Jev · TypeSafe (our credential)": {
|
|
253
|
+
"main": [
|
|
254
|
+
[
|
|
255
|
+
{
|
|
256
|
+
"node": "Judge · answers",
|
|
257
|
+
"type": "main",
|
|
258
|
+
"index": 0
|
|
259
|
+
}
|
|
260
|
+
]
|
|
261
|
+
]
|
|
262
|
+
},
|
|
263
|
+
"Judge · answers": {
|
|
264
|
+
"main": [
|
|
265
|
+
[
|
|
266
|
+
{
|
|
267
|
+
"node": "Grade + aggregate",
|
|
268
|
+
"type": "main",
|
|
269
|
+
"index": 0
|
|
270
|
+
}
|
|
271
|
+
]
|
|
272
|
+
]
|
|
273
|
+
},
|
|
274
|
+
"Grade + aggregate": {
|
|
275
|
+
"main": [
|
|
276
|
+
[
|
|
277
|
+
{
|
|
278
|
+
"node": "Record the batch to the coordination layer",
|
|
279
|
+
"type": "main",
|
|
280
|
+
"index": 0
|
|
281
|
+
}
|
|
282
|
+
]
|
|
283
|
+
]
|
|
284
|
+
}
|
|
285
|
+
},
|
|
286
|
+
"settings": {
|
|
287
|
+
"executionOrder": "v1"
|
|
288
|
+
},
|
|
289
|
+
"active": false,
|
|
290
|
+
"meta": {
|
|
291
|
+
"taifoon": {
|
|
292
|
+
"contract": "https://www.taifoon.io/schemas/coordination/v1/",
|
|
293
|
+
"steps": {
|
|
294
|
+
"Grade a batch": {
|
|
295
|
+
"in": null,
|
|
296
|
+
"out": null,
|
|
297
|
+
"does": "start a batch (pass { subjects, jobId, threshold } or run the demo set)"
|
|
298
|
+
},
|
|
299
|
+
"Build the batch": {
|
|
300
|
+
"in": null,
|
|
301
|
+
"out": "https://www.taifoon.io/schemas/coordination/v1/hire.json",
|
|
302
|
+
"does": "one canonical HIRE per subject (a delivered handshake, or a job on chain); no state is built here"
|
|
303
|
+
},
|
|
304
|
+
"Judge · prepare": {
|
|
305
|
+
"in": "https://www.taifoon.io/schemas/coordination/v1/hire.json",
|
|
306
|
+
"out": "https://www.taifoon.io/schemas/coordination/v1/facts.json",
|
|
307
|
+
"does": "POST /v1/judge/compose (prepare) per subject: the facts decide first; the pack is built from DELIVERY + FACTS; else the exact text Jev reads + prepare_digest"
|
|
308
|
+
},
|
|
309
|
+
"Hard fail?": {
|
|
310
|
+
"in": "https://www.taifoon.io/schemas/coordination/v1/facts.json",
|
|
311
|
+
"out": "https://www.taifoon.io/schemas/coordination/v1/facts.json",
|
|
312
|
+
"does": "branch: a hard fail is final (Jev is not asked) and goes straight to the batch"
|
|
313
|
+
},
|
|
314
|
+
"Jev · TypeSafe (our credential)": {
|
|
315
|
+
"in": "https://www.taifoon.io/schemas/coordination/v1/facts.json",
|
|
316
|
+
"out": "https://www.taifoon.io/schemas/coordination/v1/answers.json",
|
|
317
|
+
"does": "TypeSafe ask on the prepared text (jevStateOf): every RUBRIC_ASKED answer with its whole distribution"
|
|
318
|
+
},
|
|
319
|
+
"Judge · answers": {
|
|
320
|
+
"in": "https://www.taifoon.io/schemas/coordination/v1/answers.json",
|
|
321
|
+
"out": "https://www.taifoon.io/schemas/coordination/v1/verdict.json",
|
|
322
|
+
"does": "POST /v1/judge/compose (answers): recompose under THRESHOLDS_v1, record decision.v2 and the answers (jev.answer.v1), anchor both"
|
|
323
|
+
},
|
|
324
|
+
"Grade + aggregate": {
|
|
325
|
+
"in": [
|
|
326
|
+
"https://www.taifoon.io/schemas/coordination/v1/facts.json",
|
|
327
|
+
"https://www.taifoon.io/schemas/coordination/v1/verdict.json"
|
|
328
|
+
],
|
|
329
|
+
"out": null,
|
|
330
|
+
"does": "the batch outcome from the recorded verdicts (Conformant / NonConformant / NoGrade counts, fail fraction vs threshold) — a batch record, not a canonical entity"
|
|
331
|
+
},
|
|
332
|
+
"Record the batch to the coordination layer": {
|
|
333
|
+
"in": null,
|
|
334
|
+
"out": null,
|
|
335
|
+
"does": "POST /v1/judge/batch (relayer key): the batch record, keyed batch:<jobId>"
|
|
336
|
+
}
|
|
337
|
+
}
|
|
338
|
+
}
|
|
339
|
+
},
|
|
340
|
+
"tags": []
|
|
341
|
+
}
|