@taifoon/jev 0.2.17 → 0.2.18
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +13 -7
- package/bin/run.mjs +2 -2
- package/dist/ask.d.ts +18 -2
- package/dist/ask.js +30 -0
- package/dist/grade.d.ts +5 -0
- package/dist/grade.js +6 -3
- package/dist/pipeline.js +7 -2
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -14,7 +14,8 @@ npx @taifoon/jev run --demo
|
|
|
14
14
|
|
|
15
15
|

|
|
16
16
|
|
|
17
|
-
Then grade your own with your
|
|
17
|
+
Then grade your own: with no key, on your free grades on the Taifoon coordination layer (3 a day per caller), or with
|
|
18
|
+
your TypeSafe key from [console.typesafe.ai](https://console.typesafe.ai). Installing it also
|
|
18
19
|
installs Taifoon's other packages (@taifoon/jev-wilson, @taifoon/n8n-nodes-typesafe). No key inside, and it signs nothing. Recording a grade on chain is optional.
|
|
19
20
|
|
|
20
21
|
## Quick start
|
|
@@ -55,7 +56,7 @@ claims the evidence cannot support. Grade one with your key:
|
|
|
55
56
|
```
|
|
56
57
|
TYPESAFE_KEY=… npx @taifoon/jev run --job-file examples/jobs/invoice-extraction.json
|
|
57
58
|
npx @taifoon/jev run --demo # offline, no key: replays the Base job below and checks it against the chain
|
|
58
|
-
npx @taifoon/jev run # a live Base job from the coordination layer's queue (
|
|
59
|
+
npx @taifoon/jev run # a live Base job from the coordination layer's queue (your TYPESAFE_KEY, else the free grades)
|
|
59
60
|
```
|
|
60
61
|
|
|
61
62
|
## A Base job, graded
|
|
@@ -121,8 +122,9 @@ Safe by default: your own TypeSafe key only, nothing recorded, nothing signed or
|
|
|
121
122
|
| `--seller-record <k>/<n>` | read from the layer | Price the premium here from the seller's record (k delivered of n graded) with [`@taifoon/jev-wilson`](https://www.npmjs.com/package/@taifoon/jev-wilson): the layer's numbers, no request |
|
|
122
123
|
| `--yes` / `--json` | ask per step / text | Run every step without asking / print the whole trace as JSON |
|
|
123
124
|
|
|
124
|
-
Environment: `TYPESAFE_KEY` (your key; without it a job that needs Jev
|
|
125
|
-
|
|
125
|
+
Environment: `TYPESAFE_KEY` (your key; without it a job that needs Jev is graded on your free grades on the layer, and
|
|
126
|
+
past them it stops and says how to pay one grade; a job the code checks reject is graded either way; `--no-layer`, `--demo` and `--job-file` ask
|
|
127
|
+
no one). `TAIFOON_RELAYER_KEY` pays layer grades past the free ones from that key's balance. Exit codes: `0` done, `1` a step failed, `2` a flag was refused.
|
|
126
128
|
|
|
127
129
|
## What you must know
|
|
128
130
|
|
|
@@ -132,8 +134,12 @@ code checks reject is still graded). Exit codes: `0` done, `1` a step failed, `2
|
|
|
132
134
|
- **Code composes the verdict** under THRESHOLDS_v1: complete, reject or needs_review. needs_review ends nothing and
|
|
133
135
|
leaves room for an appeal. Above 50 USDC a complete is held for review.
|
|
134
136
|
- **Nothing is signed.** `record()` and `evaluatorCall()` return unsigned calls. Whoever holds the seat signs.
|
|
135
|
-
- **Your key
|
|
136
|
-
directly at TypeSafe
|
|
137
|
+
- **Your key, or the free grades.** With your own TypeSafe key from [console.typesafe.ai](https://console.typesafe.ai),
|
|
138
|
+
Jev is called directly at TypeSafe; the key is not stored and never appears in a receipt. Without one, `grade()`
|
|
139
|
+
claims one of your free grades on the coordination layer (`POST https://coord.taifoon.dev/v1/judge/compose`, 3 a day
|
|
140
|
+
per caller): the layer asks Jev the same four questions and this package composes the verdict from the answers, so the
|
|
141
|
+
receipt is the same shape (`via.connection: "trial"`, with the grades left). Past them the layer answers 402 with the
|
|
142
|
+
price of one grade (x402), or pass `layerKey` (a Taifoon key) to pay from its balance. `layer: false` turns this off.
|
|
137
143
|
- **What the result is.** *Graded against a published rubric by a pinned decision model, with an appeal.* It is not
|
|
138
144
|
"independently verified". An on-chain record is an attestation by whoever sent it, not a proof the answer is right.
|
|
139
145
|
|
|
@@ -211,4 +217,4 @@ In code: `verify(digest, { network: 'arbitrum' })` (a name or a chain id), `{ rp
|
|
|
211
217
|
|
|
212
218
|
Independent project. Jev and TypeSafe are products of TypeSafe AI, Inc., which does not endorse this package.
|
|
213
219
|
|
|
214
|
-
Built by Taifoon. MIT. Its runtime dependencies are Taifoon's own packages, @taifoon/jev-wilson (the pricing) and @taifoon/n8n-nodes-typesafe; nothing third-party. It contains no key. It calls Jev
|
|
220
|
+
Built by Taifoon. MIT. Its runtime dependencies are Taifoon's own packages, @taifoon/jev-wilson (the pricing) and @taifoon/n8n-nodes-typesafe; nothing third-party. It contains no key. It calls Jev with your own TypeSafe key, or without one through your free grades on the Taifoon coordination layer.
|
package/bin/run.mjs
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
// jev run — one job through the pipeline, one step at a time.
|
|
2
|
-
// jev run the first ready job on the coordination layer's queue (
|
|
2
|
+
// jev run the first ready job on the coordination layer's queue (your TYPESAFE_KEY, else the free grades)
|
|
3
3
|
// jev run --job 8453:81100 a job you name (chain:id; ids like bitagent:8453:7287 work too)
|
|
4
4
|
// jev run --evidence pack.json your own pack, no layer at all: { subject, state, delivered?, checks?, priceUsdc? }
|
|
5
5
|
// jev run --job-file job.json one agent job: { id, task, criteria[], delivered, source?, checks[] } (see examples/jobs/)
|
|
@@ -71,7 +71,7 @@ export async function run(argv) {
|
|
|
71
71
|
|
|
72
72
|
const t = createTerm({ quiet: json });
|
|
73
73
|
const rl = yes ? null : createInterface({ input: process.stdin, output: process.stdout });
|
|
74
|
-
t.info(`jev run · ${noLayer ? 'offline' : `layer ${layer ?? 'https://coord.taifoon.dev'}`} · grade on ${demo ? `Jev's recorded answers for ${rec.job.ref} (${rec.job.task})` : ansFile ? 'supplied answers' : process.env.TYPESAFE_KEY ? 'your TypeSafe key' : 'no key: set TYPESAFE_KEY, or try --demo'} · record → ${network === 'none' ? 'none (opt in with --record devnet|base|both)' : network}${process.env.TAIFOON_RELAYER_KEY ? ' + the layer' : ''}`);
|
|
74
|
+
t.info(`jev run · ${noLayer ? 'offline' : `layer ${layer ?? 'https://coord.taifoon.dev'}`} · grade on ${demo ? `Jev's recorded answers for ${rec.job.ref} (${rec.job.task})` : ansFile ? 'supplied answers' : process.env.TYPESAFE_KEY ? 'your TypeSafe key' : noLayer ? 'no key: set TYPESAFE_KEY, or try --demo' : 'your free grades on the layer (no TypeSafe key)'} · record → ${network === 'none' ? 'none (opt in with --record devnet|base|both)' : network}${process.env.TAIFOON_RELAYER_KEY ? ' + the layer' : ''}`);
|
|
75
75
|
|
|
76
76
|
const trace = await pipeline({
|
|
77
77
|
layer, job, network, protocol, priceUsdc: price ? Number(price) : undefined, sellerRecord: recM ? { k: Number(recM[1]), n: Number(recM[2]) } : undefined,
|
package/dist/ask.d.ts
CHANGED
|
@@ -6,9 +6,16 @@ export type Asked = {
|
|
|
6
6
|
answers: Record<string, Answer>;
|
|
7
7
|
model: string | null;
|
|
8
8
|
latency_ms: number | null;
|
|
9
|
-
connection: 'key';
|
|
10
|
-
trial:
|
|
9
|
+
connection: 'key' | 'trial';
|
|
10
|
+
trial: {
|
|
11
|
+
calls: number;
|
|
12
|
+
left: number;
|
|
13
|
+
} | null;
|
|
11
14
|
};
|
|
15
|
+
/** the coordination layer, where a caller without a TypeSafe key claims its free grades */
|
|
16
|
+
export declare const LAYER_URL = "https://coord.taifoon.dev";
|
|
17
|
+
/** the questions the layer's free grade answers (RUBRIC_v1 asks exactly these); another rubric needs the caller's own key */
|
|
18
|
+
export declare const LAYER_QUESTIONS: readonly string[];
|
|
12
19
|
export declare class JevError extends Error {
|
|
13
20
|
readonly status: number;
|
|
14
21
|
readonly next?: string | undefined;
|
|
@@ -22,3 +29,12 @@ export declare function askJev(a: {
|
|
|
22
29
|
model?: string;
|
|
23
30
|
fetch?: typeof fetch;
|
|
24
31
|
}): Promise<Asked>;
|
|
32
|
+
/** A free grade on the coordination layer (or one paid from a Taifoon key's balance, with `apiKey`): the layer asks Jev the
|
|
33
|
+
* four RUBRIC questions on `text` and returns the answers; this package composes the verdict from them as with a key. */
|
|
34
|
+
export declare function askLayer(a: {
|
|
35
|
+
text: string;
|
|
36
|
+
questions: readonly Question[];
|
|
37
|
+
layer?: string;
|
|
38
|
+
apiKey?: string | null;
|
|
39
|
+
fetch?: typeof fetch;
|
|
40
|
+
}): Promise<Asked>;
|
package/dist/ask.js
CHANGED
|
@@ -1,6 +1,10 @@
|
|
|
1
1
|
export const TYPESAFE_URL = 'https://api.typesafe.ai';
|
|
2
2
|
/** The model every key-path call asks for. Pinned, not `jev-latest`: a record says which model answered, and an alias can move. */
|
|
3
3
|
export const JEV_MODEL = 'jev-1.13.0';
|
|
4
|
+
/** the coordination layer, where a caller without a TypeSafe key claims its free grades */
|
|
5
|
+
export const LAYER_URL = 'https://coord.taifoon.dev';
|
|
6
|
+
/** the questions the layer's free grade answers (RUBRIC_v1 asks exactly these); another rubric needs the caller's own key */
|
|
7
|
+
export const LAYER_QUESTIONS = Object.freeze(['spec_met', 'unsupported_claim', 'ending', 'cheat_shaped']);
|
|
4
8
|
export class JevError extends Error {
|
|
5
9
|
status;
|
|
6
10
|
next;
|
|
@@ -33,3 +37,29 @@ export async function askJev(a) {
|
|
|
33
37
|
}
|
|
34
38
|
return { answers, model: j.model ?? null, latency_ms: Date.now() - started, connection: 'key', trial: null };
|
|
35
39
|
}
|
|
40
|
+
/** A free grade on the coordination layer (or one paid from a Taifoon key's balance, with `apiKey`): the layer asks Jev the
|
|
41
|
+
* four RUBRIC questions on `text` and returns the answers; this package composes the verdict from them as with a key. */
|
|
42
|
+
export async function askLayer(a) {
|
|
43
|
+
const f = a.fetch ?? fetch;
|
|
44
|
+
const started = Date.now();
|
|
45
|
+
const other = a.questions.map((q) => q.id).filter((id) => !LAYER_QUESTIONS.includes(id));
|
|
46
|
+
if (other.length)
|
|
47
|
+
throw new JevError(`the layer's free grade answers ${LAYER_QUESTIONS.join(', ')}; this rubric also asks ${other.join(', ')}: pass { key } (your TypeSafe key)`, 400, 'grade({ key: process.env.TYPESAFE_KEY })');
|
|
48
|
+
const base = (a.layer ?? LAYER_URL).replace(/\/$/, '');
|
|
49
|
+
if (!/^https:\/\//.test(base) && !/^http:\/\/(localhost|127\.0\.0\.1)(:\d+)?$/.test(base))
|
|
50
|
+
throw new JevError('the layer is only called over https', 400);
|
|
51
|
+
const r = await f(`${base}/v1/judge/compose`, { method: 'POST', headers: { 'content-type': 'application/json', 'x-taifoon-client': '@taifoon/jev', ...(a.apiKey ? { 'x-api-key': a.apiKey } : {}) }, body: JSON.stringify({ state: a.text }), signal: AbortSignal.timeout(65_000) });
|
|
52
|
+
const j = (await r.json().catch(() => ({})));
|
|
53
|
+
if (r.status === 402)
|
|
54
|
+
throw new JevError('no free grade left for this caller today: pass { key } (your TypeSafe key), a Taifoon key (layerKey), or pay one grade with x402 at POST ' + `${base}/v1/judge/compose`, 402, `${base}/v1/judge/compose`);
|
|
55
|
+
if (!r.ok)
|
|
56
|
+
throw new JevError(`the layer answered ${r.status}${j.error ? `: ${String(j.error).slice(0, 200)}` : ''}`, r.status);
|
|
57
|
+
const answers = {};
|
|
58
|
+
for (const q of a.questions) {
|
|
59
|
+
const x = j.receipt?.answers?.[q.id];
|
|
60
|
+
if (x)
|
|
61
|
+
answers[q.id] = norm(q.id, x);
|
|
62
|
+
}
|
|
63
|
+
const g = j.quota?.grades;
|
|
64
|
+
return { answers, model: j.model ?? null, latency_ms: Date.now() - started, connection: 'trial', trial: g && typeof g.free === 'number' && typeof g.left === 'number' ? { calls: g.free, left: g.left } : null };
|
|
65
|
+
}
|
package/dist/grade.d.ts
CHANGED
|
@@ -20,6 +20,11 @@ export type GradeInput = {
|
|
|
20
20
|
caller?: string;
|
|
21
21
|
/** base URL for `key` (default https://api.typesafe.ai) */
|
|
22
22
|
endpoint?: string;
|
|
23
|
+
/** with no `key` and no `answers`: the coordination layer whose free grades are claimed (default https://coord.taifoon.dev);
|
|
24
|
+
* false never calls it (then a key or answers are required) */
|
|
25
|
+
layer?: string | false;
|
|
26
|
+
/** a Taifoon key (tfr_…): a grade past the free ones is paid from its balance on the layer */
|
|
27
|
+
layerKey?: string | null;
|
|
23
28
|
fetch?: typeof fetch;
|
|
24
29
|
/** ms since epoch for the answer record; default now */
|
|
25
30
|
at?: number;
|
package/dist/grade.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
// grade(): evidence → deterministic facts → Jev's atomic questions → composed verdict → receipt.
|
|
2
|
-
import { askJev, JevError } from './ask.js';
|
|
2
|
+
import { askJev, askLayer, JevError } from './ask.js';
|
|
3
3
|
import { answersOf, buildReceipt, DEFAULT_FACTS, inputFor, packOf, rubricOf } from './receipt.js';
|
|
4
4
|
export async function grade(g) {
|
|
5
5
|
const rubric = rubricOf(g.rubric);
|
|
@@ -17,9 +17,12 @@ export async function grade(g) {
|
|
|
17
17
|
connection = 'supplied';
|
|
18
18
|
}
|
|
19
19
|
else {
|
|
20
|
-
if (!g.key)
|
|
20
|
+
if (!g.key && g.layer === false)
|
|
21
21
|
throw new JevError('pass { key } (your TypeSafe key, from console.typesafe.ai) or { answers } you already have', 400);
|
|
22
|
-
|
|
22
|
+
// _FREE_GRADES_v1_: no key of your own → the caller's free grades on the coordination layer (then x402 or a Taifoon key)
|
|
23
|
+
asked = g.key
|
|
24
|
+
? await askJev({ text: jevState, questions: rubric.asked, key: g.key, endpoint: g.endpoint, fetch: g.fetch })
|
|
25
|
+
: await askLayer({ text: jevState, questions: rubric.asked, layer: g.layer || undefined, apiKey: g.layerKey, fetch: g.fetch });
|
|
23
26
|
const missing = rubric.asked.filter((q) => !asked.answers[q.id]).map((q) => q.id);
|
|
24
27
|
if (missing.length)
|
|
25
28
|
throw new JevError(`Jev returned no valid answer for: ${missing.join(', ')}`, 502);
|
package/dist/pipeline.js
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
// pipeline(): one job through the coordination layer's steps, each step a function with a typed output.
|
|
2
2
|
// pick → evidence → facts → grade → record → evaluator → premium → verify
|
|
3
3
|
// Every step works without the layer too: pass `evidence` (your own pack) and the layer is never read; `grade` then
|
|
4
|
-
// runs on your own TypeSafe key (or answers you already have), `record` returns unsigned calls, `premium` is priced from a `sellerRecord` you pass (or skipped). With the layer
|
|
4
|
+
// runs on your own TypeSafe key (or answers you already have, or without either on the caller's free grades on the layer), `record` returns unsigned calls, `premium` is priced from a `sellerRecord` you pass (or skipped). With the layer
|
|
5
5
|
// (https://coord.taifoon.dev by default) the job comes from /v1/judge/queue, the pack from /v1/judge/evidence, the
|
|
6
6
|
// seller's terms from /v1/pools/quote, and — only with a relayer key — the answers are recorded on the layer through
|
|
7
7
|
// /v1/judge/answers/record. Nothing here signs a transaction.
|
|
@@ -156,12 +156,17 @@ export async function pipeline(o = {}) {
|
|
|
156
156
|
}
|
|
157
157
|
else if (o.key)
|
|
158
158
|
g.key = o.key;
|
|
159
|
+
// _FREE_GRADES_v1_: no key → the caller's free grades on this layer (paid from relayerKey's balance past them); offline: none
|
|
160
|
+
else {
|
|
161
|
+
g.layer = layer ?? false;
|
|
162
|
+
g.layerKey = o.relayerKey ?? null;
|
|
163
|
+
}
|
|
159
164
|
// no key: a job the code checks reject is still graded (Jev is not asked); only asking Jev needs the key
|
|
160
165
|
try {
|
|
161
166
|
t.receipt = await grade(g);
|
|
162
167
|
}
|
|
163
168
|
catch (e) {
|
|
164
|
-
if (!o.key && !o.answers && /TypeSafe key/.test(e.message))
|
|
169
|
+
if (!o.key && !o.answers && !layer && /TypeSafe key/.test(e.message))
|
|
165
170
|
throw new Error('Jev needs your TypeSafe key: set TYPESAFE_KEY (console.typesafe.ai), or pass answers you already have');
|
|
166
171
|
throw e;
|
|
167
172
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@taifoon/jev",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.18",
|
|
4
4
|
"description": "Grade AI agent jobs with TypeSafe's Jev and record the verdict on chain: code checks the facts, Jev answers a published rubric, and you get a receipt, the on-chain record and the evaluator call for ERC-8183 and Virtuals ACP. No runtime dependencies, no key inside.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"jev",
|