@cef-ai/eval 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +23 -30
- package/dist/conformance.d.ts.map +1 -1
- package/dist/conformance.js +10 -11
- package/dist/conformance.js.map +1 -1
- package/dist/index.d.ts +3 -3
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +2 -2
- package/dist/index.js.map +1 -1
- package/dist/layout.d.ts +0 -7
- package/dist/layout.d.ts.map +1 -1
- package/dist/layout.js +0 -7
- package/dist/layout.js.map +1 -1
- package/dist/storage.js +2 -2
- package/dist/vault-store.d.ts +46 -0
- package/dist/vault-store.d.ts.map +1 -0
- package/dist/vault-store.js +127 -0
- package/dist/vault-store.js.map +1 -0
- package/package.json +1 -1
- package/dist/migrate.d.ts +0 -30
- package/dist/migrate.d.ts.map +0 -1
- package/dist/migrate.js +0 -113
- package/dist/migrate.js.map +0 -1
package/README.md
CHANGED
|
@@ -61,7 +61,7 @@ never fail a case.
|
|
|
61
61
|
|
|
62
62
|
**`$regex` and trust.** Patterns run in JavaScript's backtracking engine,
|
|
63
63
|
synchronously, inside scoring — nothing can time one out, so a catastrophic
|
|
64
|
-
pattern freezes the CLI or the ROC tab. Cases come from
|
|
64
|
+
pattern freezes the CLI or the ROC tab. Cases come from shared storage, so
|
|
65
65
|
scoring refuses (fails the field with `unsafe $regex refused: …`) a pattern
|
|
66
66
|
longer than 256 characters or one that quantifies a group which itself contains
|
|
67
67
|
a quantifier (`(a+)+`, `(\w+\s?)*`). That is a heuristic, not a proof — an
|
|
@@ -84,7 +84,14 @@ front: `missing`, `typeChanged`, and `unexpectedNew` (declared, never expected).
|
|
|
84
84
|
## Storage
|
|
85
85
|
|
|
86
86
|
On an injected `EvalStore` (`list`, `get`, `put`, `putIfAbsent`; run
|
|
87
|
-
`storeConformance` against an implementation).
|
|
87
|
+
`storeConformance` against an implementation). ROC and `cef eval` keep it in
|
|
88
|
+
the vault the workflow runs in, in the workflow's scope, through the vault's
|
|
89
|
+
object API: `vaultObjectStore(vault.scope(scope).objects)` puts every key below
|
|
90
|
+
`evals/` in that scope's objects. The vault decides access the way it does for
|
|
91
|
+
running the workflow: a read grant on the scope reads datasets and experiments,
|
|
92
|
+
a write grant edits datasets and runs experiments. `putIfAbsent` is a
|
|
93
|
+
create-only upload (`If-None-Match: *`; a taken path answers 409
|
|
94
|
+
`OBJECT_EXISTS`). Keys, relative to that root:
|
|
88
95
|
|
|
89
96
|
```
|
|
90
97
|
workflows/<wf>/datasets/<dataset>/header.json Header (+ baselineId)
|
|
@@ -101,12 +108,6 @@ Experiments are siblings of the datasets: an experiment names the
|
|
|
101
108
|
`dataset@version` it ran. Every dataset function takes `(store, workflowId,
|
|
102
109
|
dataset, …)`, every experiment function `(store, workflowId, expId, …)`.
|
|
103
110
|
|
|
104
|
-
Storage written by the first release (`evals/datasets/<dataset>/…`, experiments
|
|
105
|
-
inside the dataset) is copied into this layout by `migrateLegacyLayout(store)`
|
|
106
|
-
(`cef eval migrate`): `putIfAbsent` per key, legacy keys left in place, a
|
|
107
|
-
`evals/MIGRATED.json` marker written once every dataset was copied.
|
|
108
|
-
`needsLegacyMigration(store)` says whether to run it.
|
|
109
|
-
|
|
110
111
|
## Dataset versions
|
|
111
112
|
|
|
112
113
|
Versions are cut automatically: `ensureDatasetVersion(store, wf, dataset)`
|
|
@@ -150,13 +151,14 @@ different attempts and never publish into the same contexts.
|
|
|
150
151
|
## CLI walkthrough
|
|
151
152
|
|
|
152
153
|
```sh
|
|
153
|
-
export
|
|
154
|
-
export CEF_KEYSTORE=~/owner.json # owner wallet (or CEF_SECRET_PHRASE)
|
|
154
|
+
export CEF_KEYSTORE=~/me.json # your wallet (or CEF_SECRET_PHRASE) — or CEF_ACCESS_TOKEN alone
|
|
155
155
|
export CEF_KEYSTORE_PASSWORD=…
|
|
156
|
+
export CEF_VAULT_ID=… # the vault the workflow runs in (default: the wallet's own)
|
|
156
157
|
export CEF_ACCESS_TOKEN=… # CLI access token from ROC, for the orchestrator
|
|
158
|
+
export CEF_AS_PUBKEY=0x… # the agent service — `run` only
|
|
157
159
|
|
|
158
160
|
# In the workflow's folder: --workflow defaults to the workflow cef.config.ts
|
|
159
|
-
# (or dist/) declares, --dir to ./datasets.
|
|
161
|
+
# (or dist/) declares, --scope to its requiredScopes (else "default"), --dir to ./datasets.
|
|
160
162
|
cef eval create triage --max-duration-ms 30000
|
|
161
163
|
cef eval pull triage # → datasets/triage/{header.json,cases/*.json,.versions.json}
|
|
162
164
|
$EDITOR datasets/triage/cases/refund-angry.json
|
|
@@ -166,38 +168,29 @@ cef eval run triage --workflow-version 1.5.0 --repeats 3 --baseline exp-mgfz8c1k
|
|
|
166
168
|
cef eval run triage --dataset-version 2 # an older dataset version
|
|
167
169
|
cef eval experiments triage
|
|
168
170
|
cef eval compare exp-mgfz8c1k-a1b2 exp-mgg0p2xq-9zt4
|
|
169
|
-
cef eval migrate # once, for a bucket written by the first release
|
|
170
171
|
```
|
|
171
172
|
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
Store credentials are taken in this order: explicit `--access-key`/`--secret`
|
|
179
|
-
(`$CEF_EVAL_KEY`/`$CEF_EVAL_SECRET`, an S3 key issued for the owner's wallet);
|
|
180
|
-
then the owner's wallet, minting as above. `$CEF_DDC_ACCESS_TOKEN`, the token
|
|
181
|
-
`cef push` uses, cannot open dataset storage: it is limited to the agent
|
|
182
|
-
service's registry bucket, while datasets live in the gateway bucket
|
|
183
|
-
`as-<pubkey>`, a separate DDC bucket, and it cannot be delegated to the gateway.
|
|
184
|
-
With only that token set, `cef eval` says so and names the two options above.
|
|
173
|
+
Every command reads and writes the vault: signed by the wallet when one is
|
|
174
|
+
given, else carried by `--access-token`/`$CEF_ACCESS_TOKEN` (a wallet-api
|
|
175
|
+
delegation, which needs `ddc-read`/`ddc-write`). Members of an organization's
|
|
176
|
+
vault use their own wallet or token; their grant on the scope decides what they
|
|
177
|
+
may do. `run` publishes `workflow.start` into the same scope.
|
|
185
178
|
|
|
186
179
|
`run` records the version's Result — the live deployment's graph when the
|
|
187
180
|
version is live, else the project's own graph of exactly that version — and
|
|
188
181
|
warns before starting about cases expecting fields it no longer returns. The
|
|
189
182
|
baseline stays on the platform: `pull` leaves it out of `header.json`, `push`
|
|
190
|
-
keeps the
|
|
183
|
+
keeps the vault's, and `run` compares against it when `--baseline` is not given.
|
|
191
184
|
|
|
192
185
|
Every command takes `--json`. `push` writes a new revision only for a case that
|
|
193
|
-
changed and refuses one edited in the
|
|
194
|
-
write over it). A repo case that is deleted or archived in the
|
|
186
|
+
changed and refuses one edited in the vault since the last pull (`--force` to
|
|
187
|
+
write over it). A repo case that is deleted or archived in the vault is refused
|
|
195
188
|
before anything is written; `--restore` undeletes/unarchives it and writes the
|
|
196
189
|
repo's version (`--force` does not). `pull` and `push --prune` only remove what
|
|
197
190
|
the last pull wrote (`.versions.json`): a case file the pull never wrote is
|
|
198
|
-
reported as local-only and kept, and a case added in the
|
|
191
|
+
reported as local-only and kept, and a case added in the vault after the pull
|
|
199
192
|
is never pruned. `pull` records each case's content hash in `.versions.json` and
|
|
200
193
|
refuses, before writing anything, to overwrite or remove a case file edited since
|
|
201
|
-
the last pull/push — one matching neither that hash nor the
|
|
194
|
+
the last pull/push — one matching neither that hash nor the vault's current
|
|
202
195
|
content — unless `--force`. Both `pull` and `push` refuse a dataset dir whose
|
|
203
196
|
`header.json` names another workflow.
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"conformance.d.ts","sourceRoot":"","sources":["../src/conformance.ts"],"names":[],"mappings":"AASA,OAAO,KAAK,EAAE,SAAS,EAAkB,MAAM,cAAc,CAAC;AAwC9D,wBAAgB,gBAAgB,CAC9B,SAAS,EAAE,MAAM,OAAO,CAAC,SAAS,CAAC,GAClC,KAAK,CAAC;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,GAAG,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAA;CAAE,CAAC,
|
|
1
|
+
{"version":3,"file":"conformance.d.ts","sourceRoot":"","sources":["../src/conformance.ts"],"names":[],"mappings":"AASA,OAAO,KAAK,EAAE,SAAS,EAAkB,MAAM,cAAc,CAAC;AAwC9D,wBAAgB,gBAAgB,CAC9B,SAAS,EAAE,MAAM,OAAO,CAAC,SAAS,CAAC,GAClC,KAAK,CAAC;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,GAAG,EAAE,MAAM,OAAO,CAAC,IAAI,CAAC,CAAA;CAAE,CAAC,CAwPnD"}
|
package/dist/conformance.js
CHANGED
|
@@ -115,9 +115,9 @@ export function storeConformance(makeStore) {
|
|
|
115
115
|
// the version number and the revision number are both DERIVED, so two
|
|
116
116
|
// writers compute the same one and only the conditional write decides
|
|
117
117
|
// who holds it. `memoryStore` declares `single-threaded` and passes by
|
|
118
|
-
// construction (no `await` between its check and its write);
|
|
119
|
-
//
|
|
120
|
-
//
|
|
118
|
+
// construction (no `await` between its check and its write);
|
|
119
|
+
// `vaultObjectStore` declares `conditional-write`, which vault-api backs
|
|
120
|
+
// with `If-None-Match: *` (409 OBJECT_EXISTS on a taken key).
|
|
121
121
|
name: "putIfAbsent declares its mechanism, and 8 concurrent calls leave one winner",
|
|
122
122
|
body: async (s, mechanism) => {
|
|
123
123
|
// Half of this union IS falsifiable in this process, and that half
|
|
@@ -152,7 +152,7 @@ export function storeConformance(makeStore) {
|
|
|
152
152
|
const bodies = Array.from({ length: 8 }, (_, i) => `writer-${i}`);
|
|
153
153
|
const won = await Promise.all(bodies.map((body) => s.putIfAbsent(key, body)));
|
|
154
154
|
const winners = won.filter(Boolean).length;
|
|
155
|
-
// Zero winners is not the same failure as two: if the
|
|
155
|
+
// Zero winners is not the same failure as two: if the server refused
|
|
156
156
|
// every writer (e.g. answered 409/412 to all eight, including the one
|
|
157
157
|
// that would otherwise have won), the write-once contract was never
|
|
158
158
|
// actually exercised — that's contention exhausting the race, not a
|
|
@@ -160,7 +160,7 @@ export function storeConformance(makeStore) {
|
|
|
160
160
|
// someone hunting a storage bug that isn't there.
|
|
161
161
|
if (winners === 0) {
|
|
162
162
|
throw new Error("store conformance: contention, not a broken guarantee — all 8 concurrent " +
|
|
163
|
-
"putIfAbsent calls on one key answered false (the
|
|
163
|
+
"putIfAbsent calls on one key answered false (the server refused every " +
|
|
164
164
|
"writer, e.g. 409/412 to all eight, including whichever would have won); " +
|
|
165
165
|
"nobody actually exercised write-once here, so this is not a storage bug");
|
|
166
166
|
}
|
|
@@ -203,8 +203,8 @@ export function storeConformance(makeStore) {
|
|
|
203
203
|
name: "list pages past any server limit",
|
|
204
204
|
body: async (s) => {
|
|
205
205
|
const root = k();
|
|
206
|
-
//
|
|
207
|
-
// naive caller. A set that silently stops at 1000 cuts a version
|
|
206
|
+
// An S3-style list caps a page (1000 keys) and does not always signal
|
|
207
|
+
// truncation to a naive caller. A set that silently stops at 1000 cuts a version
|
|
208
208
|
// missing the rest, and the experiment built from it looks complete.
|
|
209
209
|
// Fixed at 1050 on purpose — a consumer whose backend makes this too
|
|
210
210
|
// slow to run every commit should filter storeConformance's returned
|
|
@@ -217,7 +217,7 @@ export function storeConformance(makeStore) {
|
|
|
217
217
|
const expected = new Set();
|
|
218
218
|
// Bounded concurrency: 1050 sequential round trips over HTTP is the
|
|
219
219
|
// slow part of this check, not `list` itself — 32 in flight is fast
|
|
220
|
-
// against a real
|
|
220
|
+
// against a real server without opening 1050 sockets at once.
|
|
221
221
|
await pool(Array.from({ length: n }, (_, i) => i), 32, async (i) => {
|
|
222
222
|
const key = `${root}/${i}.json`;
|
|
223
223
|
await s.put(key, String(i));
|
|
@@ -242,9 +242,8 @@ export function storeConformance(makeStore) {
|
|
|
242
242
|
* whichever checks you keep, the store had to say how `putIfAbsent` is
|
|
243
243
|
* atomic before any of them ran.
|
|
244
244
|
*
|
|
245
|
-
* `makeStore()` is still called exactly once per check
|
|
246
|
-
*
|
|
247
|
-
* precisely because this is per-entry and not per-operation.
|
|
245
|
+
* `makeStore()` is still called exactly once per check, so a consumer can
|
|
246
|
+
* open one connection per check rather than one per operation.
|
|
248
247
|
*/
|
|
249
248
|
return checks.map(({ name, body }) => ({
|
|
250
249
|
name,
|
package/dist/conformance.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"conformance.js","sourceRoot":"","sources":["../src/conformance.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AACH,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAC;AAGjC,MAAM,MAAM,GAAG,CAAC,EAAW,EAAE,IAAY,EAAQ,EAAE;IACjD,IAAI,CAAC,EAAE;QAAE,MAAM,IAAI,KAAK,CAAC,sBAAsB,IAAI,EAAE,CAAC,CAAC;AACzD,CAAC,CAAC;AAEF,6DAA6D;AAC7D,MAAM,UAAU,GAA8B,CAAC,mBAAmB,EAAE,iBAAiB,CAAC,CAAC;AAEvF;;;;;;;;;GASG;AACH,SAAS,iBAAiB,CAAC,KAAgB;IACzC,MAAM,QAAQ,GAAY,KAAK,CAAC,SAAS,CAAC;IAC1C,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;QAC3B,MAAM,IAAI,KAAK,CACb,kFAAkF;YAChF,+EAA+E;YAC/E,+EAA+E;YAC/E,8BAA8B,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,+BAA+B;YAClF,qDAAqD,CACxD,CAAC;IACJ,CAAC;IACD,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,QAA0B,CAAC,EAAE,CAAC;QACrD,MAAM,IAAI,KAAK,CACb,4CAA4C,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,cAAc;YAChF,cAAc,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,0CAA0C;YAC7E,oCAAoC,CACvC,CAAC;IACJ,CAAC;IACD,OAAO,QAA0B,CAAC;AACpC,CAAC;AAED,MAAM,UAAU,gBAAgB,CAC9B,SAAmC;IAEnC,MAAM,CAAC,GAAG,GAAG,EAAE,CAAC,eAAe,MAAM,CAAC,UAAU,EAAE,EAAE,CAAC;IACrD,4EAA4E;IAC5E,wEAAwE;IACxE,uDAAuD;IACvD,MAAM,MAAM,GAGP;QACH;YACE,IAAI,EAAE,mDAAmD;YACzD,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,MAAM,CAAC,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,KAAK,IAAI,EAAE,sCAAsC,CAAC,CAAC;YAC9E,CAAC;SACF;QACD;YACE,IAAI,EAAE,0CAA0C;YAChD,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,MAAM,GAAG,GAAG,CAAC,EAAE,CAAC;gBAChB,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,EAAE,WAAW,CAAC,CAAC;gBAC9B,MAAM,CAAC,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,KAAK,WAAW,EAAE,iCAAiC,CAAC,CAAC;YAChF,CAAC;SACF;QACD;YACE,gEAAgE;YAChE,wEAAwE;YACxE,kEAAkE;YAClE,IAAI,EAAE,6CAA6C;YACnD,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,MAAM,GAAG,GAAG,CAAC,EAAE,CAAC;gBAChB,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC;gBAC1B,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC;gBAC3B,MAAM,CAAC,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,KAAK,QAAQ,EAAE,qCAAqC,CAAC,CAAC;YACjF,CAAC;SACF;QACD;YACE,gEAAgE;YAChE,wEAAwE;YACxE,IAAI,EAAE,2DAA2D;YACjE,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,IAAI,CAAC,CAAC,CAAC,YAAY;oBAAE,OAAO;gBAC5B,MAAM,IAAI,GAAG,GAAG,CAAC,EAAE,eAAe,CAAC;gBACnC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,yBAAyB,EAAE,GAAG,CAAC,CAAC;gBACnD,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,6BAA6B,EAAE,GAAG,CAAC,CAAC;gBACvD,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,0BAA0B,EAAE,GAAG,CAAC,CAAC;gBACpD,MAAM,GAAG,GAAG,MAAM,CAAC,CAAC,YAAY,CAAC,GAAG,IAAI,YAAY,CAAC,CAAC;gBACtD,MAAM,CACJ,IAAI,CAAC,SAAS,CAAC,CAAC,GAAG,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,KAAK,IAAI,CAAC,SAAS,CAAC,CAAC,GAAG,IAAI,cAAc,EAAE,GAAG,IAAI,cAAc,CAAC,CAAC,EAClG,4DAA4D,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,EAAE,CAClF,CAAC;YACJ,CAAC;SACF;QACD;YACE,IAAI,EAAE,uDAAuD;YAC7D,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,MAAM,GAAG,GAAG,CAAC,EAAE,CAAC;gBAChB,MAAM,CAAC,MAAM,CAAC,CAAC,WAAW,CAAC,GAAG,EAAE,OAAO,CAAC,EAAE,8BAA8B,CAAC,CAAC;gBAC1E,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,WAAW,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC,EAAE,0BAA0B,CAAC,CAAC;gBAC1E,MAAM,CACJ,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,KAAK,OAAO,EAC9B,6DAA6D,CAC9D,CAAC;YACJ,CAAC;SACF;QACD;YACE,+CAA+C;YAC/C,EAAE;YACF,mEAAmE;YACnE,sEAAsE;YACtE,uEAAuE;YACvE,sEAAsE;YACtE,qEAAqE;YACrE,yEAAyE;YACzE,qEAAqE;YACrE,wEAAwE;YACxE,qEAAqE;YACrE,yEAAyE;YACzE,qEAAqE;YACrE,qBAAqB;YACrB,EAAE;YACF,sEAAsE;YACtE,sEAAsE;YACtE,sEAAsE;YACtE,uEAAuE;YACvE,
|
|
1
|
+
{"version":3,"file":"conformance.js","sourceRoot":"","sources":["../src/conformance.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AACH,OAAO,EAAE,IAAI,EAAE,MAAM,WAAW,CAAC;AAGjC,MAAM,MAAM,GAAG,CAAC,EAAW,EAAE,IAAY,EAAQ,EAAE;IACjD,IAAI,CAAC,EAAE;QAAE,MAAM,IAAI,KAAK,CAAC,sBAAsB,IAAI,EAAE,CAAC,CAAC;AACzD,CAAC,CAAC;AAEF,6DAA6D;AAC7D,MAAM,UAAU,GAA8B,CAAC,mBAAmB,EAAE,iBAAiB,CAAC,CAAC;AAEvF;;;;;;;;;GASG;AACH,SAAS,iBAAiB,CAAC,KAAgB;IACzC,MAAM,QAAQ,GAAY,KAAK,CAAC,SAAS,CAAC;IAC1C,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;QAC3B,MAAM,IAAI,KAAK,CACb,kFAAkF;YAChF,+EAA+E;YAC/E,+EAA+E;YAC/E,8BAA8B,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,+BAA+B;YAClF,qDAAqD,CACxD,CAAC;IACJ,CAAC;IACD,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,QAA0B,CAAC,EAAE,CAAC;QACrD,MAAM,IAAI,KAAK,CACb,4CAA4C,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,cAAc;YAChF,cAAc,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,0CAA0C;YAC7E,oCAAoC,CACvC,CAAC;IACJ,CAAC;IACD,OAAO,QAA0B,CAAC;AACpC,CAAC;AAED,MAAM,UAAU,gBAAgB,CAC9B,SAAmC;IAEnC,MAAM,CAAC,GAAG,GAAG,EAAE,CAAC,eAAe,MAAM,CAAC,UAAU,EAAE,EAAE,CAAC;IACrD,4EAA4E;IAC5E,wEAAwE;IACxE,uDAAuD;IACvD,MAAM,MAAM,GAGP;QACH;YACE,IAAI,EAAE,mDAAmD;YACzD,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,MAAM,CAAC,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,KAAK,IAAI,EAAE,sCAAsC,CAAC,CAAC;YAC9E,CAAC;SACF;QACD;YACE,IAAI,EAAE,0CAA0C;YAChD,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,MAAM,GAAG,GAAG,CAAC,EAAE,CAAC;gBAChB,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,EAAE,WAAW,CAAC,CAAC;gBAC9B,MAAM,CAAC,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,KAAK,WAAW,EAAE,iCAAiC,CAAC,CAAC;YAChF,CAAC;SACF;QACD;YACE,gEAAgE;YAChE,wEAAwE;YACxE,kEAAkE;YAClE,IAAI,EAAE,6CAA6C;YACnD,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,MAAM,GAAG,GAAG,CAAC,EAAE,CAAC;gBAChB,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC;gBAC1B,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC;gBAC3B,MAAM,CAAC,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,KAAK,QAAQ,EAAE,qCAAqC,CAAC,CAAC;YACjF,CAAC;SACF;QACD;YACE,gEAAgE;YAChE,wEAAwE;YACxE,IAAI,EAAE,2DAA2D;YACjE,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,IAAI,CAAC,CAAC,CAAC,YAAY;oBAAE,OAAO;gBAC5B,MAAM,IAAI,GAAG,GAAG,CAAC,EAAE,eAAe,CAAC;gBACnC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,yBAAyB,EAAE,GAAG,CAAC,CAAC;gBACnD,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,6BAA6B,EAAE,GAAG,CAAC,CAAC;gBACvD,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,0BAA0B,EAAE,GAAG,CAAC,CAAC;gBACpD,MAAM,GAAG,GAAG,MAAM,CAAC,CAAC,YAAY,CAAC,GAAG,IAAI,YAAY,CAAC,CAAC;gBACtD,MAAM,CACJ,IAAI,CAAC,SAAS,CAAC,CAAC,GAAG,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,KAAK,IAAI,CAAC,SAAS,CAAC,CAAC,GAAG,IAAI,cAAc,EAAE,GAAG,IAAI,cAAc,CAAC,CAAC,EAClG,4DAA4D,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,EAAE,CAClF,CAAC;YACJ,CAAC;SACF;QACD;YACE,IAAI,EAAE,uDAAuD;YAC7D,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,MAAM,GAAG,GAAG,CAAC,EAAE,CAAC;gBAChB,MAAM,CAAC,MAAM,CAAC,CAAC,WAAW,CAAC,GAAG,EAAE,OAAO,CAAC,EAAE,8BAA8B,CAAC,CAAC;gBAC1E,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,WAAW,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC,EAAE,0BAA0B,CAAC,CAAC;gBAC1E,MAAM,CACJ,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,KAAK,OAAO,EAC9B,6DAA6D,CAC9D,CAAC;YACJ,CAAC;SACF;QACD;YACE,+CAA+C;YAC/C,EAAE;YACF,mEAAmE;YACnE,sEAAsE;YACtE,uEAAuE;YACvE,sEAAsE;YACtE,qEAAqE;YACrE,yEAAyE;YACzE,qEAAqE;YACrE,wEAAwE;YACxE,qEAAqE;YACrE,yEAAyE;YACzE,qEAAqE;YACrE,qBAAqB;YACrB,EAAE;YACF,sEAAsE;YACtE,sEAAsE;YACtE,sEAAsE;YACtE,uEAAuE;YACvE,6DAA6D;YAC7D,yEAAyE;YACzE,8DAA8D;YAC9D,IAAI,EAAE,6EAA6E;YACnF,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,SAAS,EAAE,EAAE;gBAC3B,mEAAmE;gBACnE,mCAAmC;gBACnC,EAAE;gBACF,mEAAmE;gBACnE,sEAAsE;gBACtE,sEAAsE;gBACtE,uEAAuE;gBACvE,8DAA8D;gBAC9D,qEAAqE;gBACrE,wDAAwD;gBACxD,mEAAmE;gBACnE,oEAAoE;gBACpE,sEAAsE;gBACtE,uEAAuE;gBACvE,kEAAkE;gBAClE,6BAA6B;gBAC7B,IAAI,SAAS,KAAK,iBAAiB,EAAE,CAAC;oBACpC,MAAM,GAAG,GAAG,CAAC,EAAE,CAAC;oBAChB,MAAM,QAAQ,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC;wBACjC,CAAC,CAAC,WAAW,CAAC,GAAG,EAAE,aAAa,CAAC;wBACjC,CAAC,CAAC,WAAW,CAAC,GAAG,EAAE,aAAa,CAAC;qBAClC,CAAC,CAAC;oBACH,MAAM,CACJ,QAAQ,CAAC,CAAC,CAAC,KAAK,IAAI,IAAI,QAAQ,CAAC,CAAC,CAAC,KAAK,KAAK,EAC7C,gFAAgF;wBAC9E,0EAA0E;wBAC1E,GAAG,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,mDAAmD;wBAC9E,oCAAoC,CACvC,CAAC;oBACF,MAAM,CACJ,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,KAAK,aAAa,EACpC,sDAAsD,CACvD,CAAC;gBACJ,CAAC;gBACD,MAAM,GAAG,GAAG,CAAC,EAAE,CAAC;gBAChB,MAAM,MAAM,GAAG,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,EAAE,CAAC,CAAC;gBAClE,MAAM,GAAG,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC,CAAC,WAAW,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC,CAAC,CAAC;gBAC9E,MAAM,OAAO,GAAG,GAAG,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,MAAM,CAAC;gBAC3C,qEAAqE;gBACrE,sEAAsE;gBACtE,oEAAoE;gBACpE,oEAAoE;gBACpE,oEAAoE;gBACpE,kDAAkD;gBAClD,IAAI,OAAO,KAAK,CAAC,EAAE,CAAC;oBAClB,MAAM,IAAI,KAAK,CACb,2EAA2E;wBACzE,wEAAwE;wBACxE,0EAA0E;wBAC1E,yEAAyE,CAC5E,CAAC;gBACJ,CAAC;gBACD,MAAM,CACJ,OAAO,KAAK,CAAC,EACb,kEAAkE,OAAO,MAAM,CAChF,CAAC;gBACF,MAAM,MAAM,GAAG,MAAM,CAAC,GAAG,CAAC,OAAO,CAAC,IAAI,CAAC,CAAE,CAAC;gBAC1C,MAAM,MAAM,GAAG,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;gBAChC,MAAM,CACJ,MAAM,KAAK,MAAM,EACjB,0CAA0C,MAAM,UAAU,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,EAAE,CACnF,CAAC;YACJ,CAAC;SACF;QACD;YACE,IAAI,EAAE,4DAA4D;YAClE,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,MAAM,IAAI,GAAG,CAAC,EAAE,CAAC;gBACjB,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,SAAS,EAAE,GAAG,CAAC,CAAC;gBACnC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,gBAAgB,EAAE,GAAG,CAAC,CAAC;gBAC1C,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,iBAAiB,EAAE,GAAG,CAAC,CAAC;gBAC3C,MAAM,IAAI,GAAG,CAAC,MAAM,CAAC,CAAC,IAAI,CAAC,GAAG,IAAI,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;gBAC/C,MAAM,CACJ,IAAI,CAAC,MAAM,KAAK,CAAC,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,SAAS,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,gBAAgB,EACxF,mDAAmD,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,EAAE,CAC1E,CAAC;YACJ,CAAC;SACF;QACD;YACE,sEAAsE;YACtE,qEAAqE;YACrE,6DAA6D;YAC7D,IAAI,EAAE,sEAAsE;YAC5E,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,IAAI,CAAC,CAAC,CAAC,YAAY;oBAAE,OAAO;gBAC5B,MAAM,IAAI,GAAG,CAAC,EAAE,CAAC;gBACjB,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,WAAW,EAAE,GAAG,CAAC,CAAC;gBACrC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,WAAW,EAAE,GAAG,CAAC,CAAC;gBACrC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,WAAW,EAAE,GAAG,CAAC,CAAC;gBACrC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,gBAAgB,EAAE,GAAG,CAAC,CAAC;gBAC1C,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,IAAI,mBAAmB,EAAE,GAAG,CAAC,CAAC;gBAC7C,MAAM,GAAG,GAAG,MAAM,CAAC,CAAC,YAAY,CAAC,GAAG,IAAI,GAAG,CAAC,CAAC;gBAC7C,MAAM,CACJ,IAAI,CAAC,SAAS,CAAC,CAAC,GAAG,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC,KAAK,IAAI,CAAC,SAAS,CAAC,CAAC,GAAG,IAAI,KAAK,EAAE,GAAG,IAAI,KAAK,CAAC,CAAC,EAChF,uEAAuE,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,EAAE,CAC7F,CAAC;YACJ,CAAC;SACF;QACD;YACE,IAAI,EAAE,kCAAkC;YACxC,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,EAAE;gBAChB,MAAM,IAAI,GAAG,CAAC,EAAE,CAAC;gBACjB,sEAAsE;gBACtE,iFAAiF;gBACjF,qEAAqE;gBACrE,qEAAqE;gBACrE,qEAAqE;gBACrE,oEAAoE;gBACpE,mEAAmE;gBACnE,oEAAoE;gBACpE,oEAAoE;gBACpE,uBAAuB;gBACvB,MAAM,CAAC,GAAG,IAAI,CAAC;gBACf,MAAM,QAAQ,GAAG,IAAI,GAAG,EAAU,CAAC;gBACnC,oEAAoE;gBACpE,oEAAoE;gBACpE,8DAA8D;gBAC9D,MAAM,IAAI,CACR,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EACtC,EAAE,EACF,KAAK,EAAE,CAAC,EAAE,EAAE;oBACV,MAAM,GAAG,GAAG,GAAG,IAAI,IAAI,CAAC,OAAO,CAAC;oBAChC,MAAM,CAAC,CAAC,GAAG,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC;oBAC5B,QAAQ,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;gBACpB,CAAC,CACF,CAAC;gBACF,MAAM,IAAI,GAAG,MAAM,CAAC,CAAC,IAAI,CAAC,GAAG,IAAI,GAAG,CAAC,CAAC;gBACtC,MAAM,CAAC,IAAI,CAAC,MAAM,KAAK,CAAC,EAAE,YAAY,CAAC,cAAc,IAAI,CAAC,MAAM,EAAE,CAAC,CAAC;gBACpE,MAAM,CAAC,IAAI,GAAG,CAAC,IAAI,CAAC,CAAC,IAAI,KAAK,CAAC,EAAE,kDAAkD,CAAC,CAAC;gBACrF,MAAM,CACJ,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,QAAQ,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,EACtC,qEAAqE,CACtE,CAAC;YACJ,CAAC;SACF;KACF,CAAC;IAEF;;;;;;;;;;;;;;OAcG;IACH,OAAO,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,EAAE,IAAI,EAAE,EAAE,EAAE,CAAC,CAAC;QACrC,IAAI;QACJ,GAAG,EAAE,KAAK,IAAI,EAAE;YACd,MAAM,CAAC,GAAG,MAAM,SAAS,EAAE,CAAC;YAC5B,MAAM,IAAI,CAAC,CAAC,EAAE,iBAAiB,CAAC,CAAC,CAAC,CAAC,CAAC;QACtC,CAAC;KACF,CAAC,CAAC,CAAC;AACN,CAAC"}
|
package/dist/index.d.ts
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
export type { Json, Limits, Header, Matcher, Case, VersionEntry, VersionManifest, Metrics, Score, RunStatus, RunRecord, ExperimentStatus, Experiment, Summary, FieldAccuracy, DatasetDiff, ResultField, ResultFieldType, RunEnd, WorkflowTarget, } from "./types.js";
|
|
2
|
-
export { WORKFLOWS_PREFIX,
|
|
2
|
+
export { WORKFLOWS_PREFIX, isWorkflowId, isDatasetName, isCaseId, isExperimentId, assertWorkflowId, assertDatasetName, assertCaseId, assertExperimentId, workflowPrefix, datasetsPrefix, datasetPrefix, headerKey, casesPrefix, caseDirPrefix, caseRevKey, tombstoneKey, archivedPrefix, archiveKey, versionsPrefix, versionKey, experimentsPrefix, experimentPrefix, experimentMetaKey, runsPrefix, runKey, attemptsPrefix, attemptKey, attemptFromKey, revFromKey, caseIdFromKey, archivedIdFromKey, versionFromKey, experimentIdFromMetaKey, runFromKey, canonicalJson, sha256Hex, newExperimentId, runContext, experimentContextPrefix, runIdOf, } from "./layout.js";
|
|
3
3
|
export { EvalContractError, isObject, parseHeader, parseCase, parseLimits, parseVersionManifest, parseRunRecord, parseExperiment, } from "./contract.js";
|
|
4
4
|
export { memoryStore, createDataset, getHeader, requireHeader, updateHeader, setBaseline, listWorkflows, listDatasets, listCaseIds, listArchivedCaseIds, listDeletedCaseIds, listCaseRevisions, getCaseRevision, getCase, saveCase, addCase, archiveCase, unarchiveCase, deleteCase, undeleteCase, loadWorkingSet, listVersions, latestVersionNumber, getVersion, cutVersion, diffVersionEntries, autoVersionNote, ensureDatasetVersion, versionHistory, loadVersion, putExperiment, claimAttempt, getExperiment, listExperiments, putRunRecord, getRunRecord, listRunRecords, loadExperiment, findExperiment, childPrefixes, } from "./storage.js";
|
|
5
5
|
export type { EvalStore, StoreAtomicity, CreateDatasetInput, DatasetSummary, VersionHistoryEntry } from "./storage.js";
|
|
6
|
-
export {
|
|
7
|
-
export type {
|
|
6
|
+
export { CONDITIONAL_WRITE_PROBE_KEY, ConditionalWriteUnsupportedError, vaultObjectStore } from "./vault-store.js";
|
|
7
|
+
export type { VaultObjects, VaultObjectStoreOptions } from "./vault-store.js";
|
|
8
8
|
export { resultFieldsFromGraph } from "./result.js";
|
|
9
9
|
export { storeConformance } from "./conformance.js";
|
|
10
10
|
export { FIELD_SCORER, LIMIT_SCORER, readField, deepEqual, parseBand, isMatcher, matchValue, scoreFields, effectiveLimits, scoreLimits, casePass, expectedTypesOf, checkResultCompatibility, unsafeRegex, MAX_REGEX_LENGTH, } from "./score.js";
|
package/dist/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,YAAY,EACV,IAAI,EACJ,MAAM,EACN,MAAM,EACN,OAAO,EACP,IAAI,EACJ,YAAY,EACZ,eAAe,EACf,OAAO,EACP,KAAK,EACL,SAAS,EACT,SAAS,EACT,gBAAgB,EAChB,UAAU,EACV,OAAO,EACP,aAAa,EACb,WAAW,EACX,WAAW,EACX,eAAe,EACf,MAAM,EACN,cAAc,GACf,MAAM,YAAY,CAAC;AAEpB,OAAO,EACL,gBAAgB,EAChB,
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,YAAY,EACV,IAAI,EACJ,MAAM,EACN,MAAM,EACN,OAAO,EACP,IAAI,EACJ,YAAY,EACZ,eAAe,EACf,OAAO,EACP,KAAK,EACL,SAAS,EACT,SAAS,EACT,gBAAgB,EAChB,UAAU,EACV,OAAO,EACP,aAAa,EACb,WAAW,EACX,WAAW,EACX,eAAe,EACf,MAAM,EACN,cAAc,GACf,MAAM,YAAY,CAAC;AAEpB,OAAO,EACL,gBAAgB,EAChB,YAAY,EACZ,aAAa,EACb,QAAQ,EACR,cAAc,EACd,gBAAgB,EAChB,iBAAiB,EACjB,YAAY,EACZ,kBAAkB,EAClB,cAAc,EACd,cAAc,EACd,aAAa,EACb,SAAS,EACT,WAAW,EACX,aAAa,EACb,UAAU,EACV,YAAY,EACZ,cAAc,EACd,UAAU,EACV,cAAc,EACd,UAAU,EACV,iBAAiB,EACjB,gBAAgB,EAChB,iBAAiB,EACjB,UAAU,EACV,MAAM,EACN,cAAc,EACd,UAAU,EACV,cAAc,EACd,UAAU,EACV,aAAa,EACb,iBAAiB,EACjB,cAAc,EACd,uBAAuB,EACvB,UAAU,EACV,aAAa,EACb,SAAS,EACT,eAAe,EACf,UAAU,EACV,uBAAuB,EACvB,OAAO,GACR,MAAM,aAAa,CAAC;AAErB,OAAO,EACL,iBAAiB,EACjB,QAAQ,EACR,WAAW,EACX,SAAS,EACT,WAAW,EACX,oBAAoB,EACpB,cAAc,EACd,eAAe,GAChB,MAAM,eAAe,CAAC;AAEvB,OAAO,EACL,WAAW,EACX,aAAa,EACb,SAAS,EACT,aAAa,EACb,YAAY,EACZ,WAAW,EACX,aAAa,EACb,YAAY,EACZ,WAAW,EACX,mBAAmB,EACnB,kBAAkB,EAClB,iBAAiB,EACjB,eAAe,EACf,OAAO,EACP,QAAQ,EACR,OAAO,EACP,WAAW,EACX,aAAa,EACb,UAAU,EACV,YAAY,EACZ,cAAc,EACd,YAAY,EACZ,mBAAmB,EACnB,UAAU,EACV,UAAU,EACV,kBAAkB,EAClB,eAAe,EACf,oBAAoB,EACpB,cAAc,EACd,WAAW,EACX,aAAa,EACb,YAAY,EACZ,aAAa,EACb,eAAe,EACf,YAAY,EACZ,YAAY,EACZ,cAAc,EACd,cAAc,EACd,cAAc,EACd,aAAa,GACd,MAAM,cAAc,CAAC;AACtB,YAAY,EAAE,SAAS,EAAE,cAAc,EAAE,kBAAkB,EAAE,cAAc,EAAE,mBAAmB,EAAE,MAAM,cAAc,CAAC;AAEvH,OAAO,EAAE,2BAA2B,EAAE,gCAAgC,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AACnH,YAAY,EAAE,YAAY,EAAE,uBAAuB,EAAE,MAAM,kBAAkB,CAAC;AAE9E,OAAO,EAAE,qBAAqB,EAAE,MAAM,aAAa,CAAC;AAEpD,OAAO,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AAEpD,OAAO,EACL,YAAY,EACZ,YAAY,EACZ,SAAS,EACT,SAAS,EACT,SAAS,EACT,SAAS,EACT,UAAU,EACV,WAAW,EACX,eAAe,EACf,WAAW,EACX,QAAQ,EACR,eAAe,EACf,wBAAwB,EACxB,WAAW,EACX,gBAAgB,GACjB,MAAM,YAAY,CAAC;AACpB,YAAY,EAAE,MAAM,EAAE,mBAAmB,EAAE,MAAM,YAAY,CAAC;AAE9D,OAAO,EAAE,YAAY,EAAE,UAAU,EAAE,SAAS,EAAE,kBAAkB,EAAE,MAAM,cAAc,CAAC;AACvF,YAAY,EACV,MAAM,EAAE,WAAW,EAAE,aAAa,EAAE,cAAc,EAAE,oBAAoB,EAAE,cAAc,GACzF,MAAM,cAAc,CAAC;AAEtB,OAAO,EAAE,aAAa,EAAE,MAAM,EAAE,kBAAkB,EAAE,mBAAmB,EAAE,MAAM,UAAU,CAAC;AAC1F,YAAY,EAAE,oBAAoB,EAAE,kBAAkB,EAAE,MAAM,UAAU,CAAC;AAEzE,OAAO,EAAE,WAAW,EAAE,eAAe,EAAE,UAAU,EAAE,UAAU,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAClG,YAAY,EAAE,WAAW,EAAE,MAAM,cAAc,CAAC"}
|
package/dist/index.js
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
|
-
export { WORKFLOWS_PREFIX,
|
|
1
|
+
export { WORKFLOWS_PREFIX, isWorkflowId, isDatasetName, isCaseId, isExperimentId, assertWorkflowId, assertDatasetName, assertCaseId, assertExperimentId, workflowPrefix, datasetsPrefix, datasetPrefix, headerKey, casesPrefix, caseDirPrefix, caseRevKey, tombstoneKey, archivedPrefix, archiveKey, versionsPrefix, versionKey, experimentsPrefix, experimentPrefix, experimentMetaKey, runsPrefix, runKey, attemptsPrefix, attemptKey, attemptFromKey, revFromKey, caseIdFromKey, archivedIdFromKey, versionFromKey, experimentIdFromMetaKey, runFromKey, canonicalJson, sha256Hex, newExperimentId, runContext, experimentContextPrefix, runIdOf, } from "./layout.js";
|
|
2
2
|
export { EvalContractError, isObject, parseHeader, parseCase, parseLimits, parseVersionManifest, parseRunRecord, parseExperiment, } from "./contract.js";
|
|
3
3
|
export { memoryStore, createDataset, getHeader, requireHeader, updateHeader, setBaseline, listWorkflows, listDatasets, listCaseIds, listArchivedCaseIds, listDeletedCaseIds, listCaseRevisions, getCaseRevision, getCase, saveCase, addCase, archiveCase, unarchiveCase, deleteCase, undeleteCase, loadWorkingSet, listVersions, latestVersionNumber, getVersion, cutVersion, diffVersionEntries, autoVersionNote, ensureDatasetVersion, versionHistory, loadVersion, putExperiment, claimAttempt, getExperiment, listExperiments, putRunRecord, getRunRecord, listRunRecords, loadExperiment, findExperiment, childPrefixes, } from "./storage.js";
|
|
4
|
-
export {
|
|
4
|
+
export { CONDITIONAL_WRITE_PROBE_KEY, ConditionalWriteUnsupportedError, vaultObjectStore } from "./vault-store.js";
|
|
5
5
|
export { resultFieldsFromGraph } from "./result.js";
|
|
6
6
|
export { storeConformance } from "./conformance.js";
|
|
7
7
|
export { FIELD_SCORER, LIMIT_SCORER, readField, deepEqual, parseBand, isMatcher, matchValue, scoreFields, effectiveLimits, scoreLimits, casePass, expectedTypesOf, checkResultCompatibility, unsafeRegex, MAX_REGEX_LENGTH, } from "./score.js";
|
package/dist/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAuBA,OAAO,EACL,gBAAgB,EAChB,
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAuBA,OAAO,EACL,gBAAgB,EAChB,YAAY,EACZ,aAAa,EACb,QAAQ,EACR,cAAc,EACd,gBAAgB,EAChB,iBAAiB,EACjB,YAAY,EACZ,kBAAkB,EAClB,cAAc,EACd,cAAc,EACd,aAAa,EACb,SAAS,EACT,WAAW,EACX,aAAa,EACb,UAAU,EACV,YAAY,EACZ,cAAc,EACd,UAAU,EACV,cAAc,EACd,UAAU,EACV,iBAAiB,EACjB,gBAAgB,EAChB,iBAAiB,EACjB,UAAU,EACV,MAAM,EACN,cAAc,EACd,UAAU,EACV,cAAc,EACd,UAAU,EACV,aAAa,EACb,iBAAiB,EACjB,cAAc,EACd,uBAAuB,EACvB,UAAU,EACV,aAAa,EACb,SAAS,EACT,eAAe,EACf,UAAU,EACV,uBAAuB,EACvB,OAAO,GACR,MAAM,aAAa,CAAC;AAErB,OAAO,EACL,iBAAiB,EACjB,QAAQ,EACR,WAAW,EACX,SAAS,EACT,WAAW,EACX,oBAAoB,EACpB,cAAc,EACd,eAAe,GAChB,MAAM,eAAe,CAAC;AAEvB,OAAO,EACL,WAAW,EACX,aAAa,EACb,SAAS,EACT,aAAa,EACb,YAAY,EACZ,WAAW,EACX,aAAa,EACb,YAAY,EACZ,WAAW,EACX,mBAAmB,EACnB,kBAAkB,EAClB,iBAAiB,EACjB,eAAe,EACf,OAAO,EACP,QAAQ,EACR,OAAO,EACP,WAAW,EACX,aAAa,EACb,UAAU,EACV,YAAY,EACZ,cAAc,EACd,YAAY,EACZ,mBAAmB,EACnB,UAAU,EACV,UAAU,EACV,kBAAkB,EAClB,eAAe,EACf,oBAAoB,EACpB,cAAc,EACd,WAAW,EACX,aAAa,EACb,YAAY,EACZ,aAAa,EACb,eAAe,EACf,YAAY,EACZ,YAAY,EACZ,cAAc,EACd,cAAc,EACd,cAAc,EACd,aAAa,GACd,MAAM,cAAc,CAAC;AAGtB,OAAO,EAAE,2BAA2B,EAAE,gCAAgC,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AAGnH,OAAO,EAAE,qBAAqB,EAAE,MAAM,aAAa,CAAC;AAEpD,OAAO,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AAEpD,OAAO,EACL,YAAY,EACZ,YAAY,EACZ,SAAS,EACT,SAAS,EACT,SAAS,EACT,SAAS,EACT,UAAU,EACV,WAAW,EACX,eAAe,EACf,WAAW,EACX,QAAQ,EACR,eAAe,EACf,wBAAwB,EACxB,WAAW,EACX,gBAAgB,GACjB,MAAM,YAAY,CAAC;AAGpB,OAAO,EAAE,YAAY,EAAE,UAAU,EAAE,SAAS,EAAE,kBAAkB,EAAE,MAAM,cAAc,CAAC;AAKvF,OAAO,EAAE,aAAa,EAAE,MAAM,EAAE,kBAAkB,EAAE,mBAAmB,EAAE,MAAM,UAAU,CAAC;AAG1F,OAAO,EAAE,WAAW,EAAE,eAAe,EAAE,UAAU,EAAE,UAAU,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC"}
|
package/dist/layout.d.ts
CHANGED
|
@@ -13,9 +13,6 @@
|
|
|
13
13
|
* workflows/<wf>/experiments/<expId>/runs/<caseId>.<repeat>.json RunRecord (write-once)
|
|
14
14
|
* workflows/<wf>/experiments/<expId>/attempts/a<n>.json attempt claim (write-once)
|
|
15
15
|
*
|
|
16
|
-
* The v1 layout (`evals/datasets/<dataset>/…`, experiments nested in their
|
|
17
|
-
* dataset) is read only by `migrateLegacyLayout`.
|
|
18
|
-
*
|
|
19
16
|
* WRITE-ONCE REVISIONS, NOT ONE FILE. Editing a case writes `r<n+1>`; nothing
|
|
20
17
|
* is rewritten, so a version that pinned `r3` by sha256 keeps loading exactly
|
|
21
18
|
* what it pinned however often the case is edited. Revision and version numbers
|
|
@@ -28,10 +25,6 @@
|
|
|
28
25
|
* to load. A marker beside it leaves history intact.
|
|
29
26
|
*/
|
|
30
27
|
export declare const WORKFLOWS_PREFIX = "workflows/";
|
|
31
|
-
/** The v1 layout's root; only `migrateLegacyLayout` reads it. */
|
|
32
|
-
export declare const LEGACY_EVALS_PREFIX = "evals/datasets/";
|
|
33
|
-
/** Written once `migrateLegacyLayout` has copied every legacy dataset. */
|
|
34
|
-
export declare const MIGRATION_MARKER_KEY = "evals/MIGRATED.json";
|
|
35
28
|
/** A workflow id is the workflow agent's alias; it is a key segment here. */
|
|
36
29
|
export declare function isWorkflowId(id: string): boolean;
|
|
37
30
|
export declare function isDatasetName(name: string): boolean;
|
package/dist/layout.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"layout.d.ts","sourceRoot":"","sources":["../src/layout.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"layout.d.ts","sourceRoot":"","sources":["../src/layout.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AAEH,eAAO,MAAM,gBAAgB,eAAe,CAAC;AAO7C,6EAA6E;AAC7E,wBAAgB,YAAY,CAAC,EAAE,EAAE,MAAM,GAAG,OAAO,CAEhD;AAED,wBAAgB,aAAa,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAEnD;AAED,wBAAgB,QAAQ,CAAC,EAAE,EAAE,MAAM,GAAG,OAAO,CAE5C;AAED,wBAAgB,cAAc,CAAC,EAAE,EAAE,MAAM,GAAG,OAAO,CAElD;AAED,wBAAgB,gBAAgB,CAAC,EAAE,EAAE,MAAM,GAAG,IAAI,CAEjD;AAED,wBAAgB,iBAAiB,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAIpD;AAED,wBAAgB,YAAY,CAAC,EAAE,EAAE,MAAM,GAAG,IAAI,CAE7C;AAED,wBAAgB,kBAAkB,CAAC,EAAE,EAAE,MAAM,GAAG,IAAI,CAEnD;AAED,eAAO,MAAM,cAAc,GAAI,YAAY,MAAM,KAAG,MAA6C,CAAC;AAClG,eAAO,MAAM,cAAc,GAAI,YAAY,MAAM,KAAG,MAAkD,CAAC;AACvG,eAAO,MAAM,aAAa,GAAI,YAAY,MAAM,EAAE,SAAS,MAAM,KAAG,MAAoD,CAAC;AACzH,eAAO,MAAM,SAAS,GAAI,YAAY,MAAM,EAAE,SAAS,MAAM,KAAG,MAA4D,CAAC;AAC7H,eAAO,MAAM,WAAW,GAAI,YAAY,MAAM,EAAE,SAAS,MAAM,KAAG,MAAuD,CAAC;AAC1H,eAAO,MAAM,aAAa,GAAI,YAAY,MAAM,EAAE,SAAS,MAAM,EAAE,QAAQ,MAAM,KAAG,MACnC,CAAC;AAClD,eAAO,MAAM,UAAU,GAAI,YAAY,MAAM,EAAE,SAAS,MAAM,EAAE,QAAQ,MAAM,EAAE,KAAK,MAAM,KAAG,MACjC,CAAC;AAC9D,eAAO,MAAM,YAAY,GAAI,YAAY,MAAM,EAAE,SAAS,MAAM,EAAE,QAAQ,MAAM,KAAG,MACtB,CAAC;AAC9D,eAAO,MAAM,cAAc,GAAI,YAAY,MAAM,EAAE,SAAS,MAAM,KAAG,MAA0D,CAAC;AAChI,eAAO,MAAM,UAAU,GAAI,YAAY,MAAM,EAAE,SAAS,MAAM,EAAE,QAAQ,MAAM,KAAG,MACzB,CAAC;AACzD,eAAO,MAAM,cAAc,GAAI,YAAY,MAAM,EAAE,SAAS,MAAM,KAAG,MAA0D,CAAC;AAChI,eAAO,MAAM,UAAU,GAAI,YAAY,MAAM,EAAE,SAAS,MAAM,EAAE,SAAS,MAAM,KAAG,MACxB,CAAC;AAC3D,eAAO,MAAM,iBAAiB,GAAI,YAAY,MAAM,KAAG,MAAqD,CAAC;AAC7G,eAAO,MAAM,gBAAgB,GAAI,YAAY,MAAM,EAAE,OAAO,MAAM,KAAG,MAAqD,CAAC;AAC3H,eAAO,MAAM,iBAAiB,GAAI,YAAY,MAAM,EAAE,OAAO,MAAM,KAAG,MACnB,CAAC;AACpD,eAAO,MAAM,UAAU,GAAI,YAAY,MAAM,EAAE,OAAO,MAAM,KAAG,MAAuD,CAAC;AACvH,eAAO,MAAM,MAAM,GAAI,YAAY,MAAM,EAAE,OAAO,MAAM,EAAE,QAAQ,MAAM,EAAE,QAAQ,MAAM,KAAG,MAC/B,CAAC;AAC7D,eAAO,MAAM,cAAc,GAAI,YAAY,MAAM,EAAE,OAAO,MAAM,KAAG,MAA2D,CAAC;AAC/H,eAAO,MAAM,UAAU,GAAI,YAAY,MAAM,EAAE,OAAO,MAAM,EAAE,SAAS,MAAM,KAAG,MACxB,CAAC;AAEzD,+EAA+E;AAC/E,wBAAgB,cAAc,CAAC,UAAU,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAK5F;AAED,0FAA0F;AAC1F,wBAAgB,UAAU,CAAC,UAAU,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAK1G;AAED,mEAAmE;AACnE,wBAAgB,aAAa,CAAC,UAAU,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAQ7F;AAED,wBAAgB,iBAAiB,CAAC,UAAU,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAKjG;AAED,wBAAgB,cAAc,CAAC,UAAU,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAK9F;AAED,wBAAgB,uBAAuB,CAAC,UAAU,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAKtF;AAED,yDAAyD;AACzD,wBAAgB,UAAU,CAAC,UAAU,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,GAAG;IAAE,MAAM,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE,MAAM,CAAA;CAAE,GAAG,IAAI,CAUpH;AAED,2DAA2D;AAC3D,wBAAgB,OAAO,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE,KAAK,EAAE,CAAC,GAAG,EAAE,MAAM,KAAK,MAAM,GAAG,IAAI,GAAG,MAAM,CAOrF;AAED;;;;GAIG;AACH,wBAAgB,aAAa,CAAC,KAAK,EAAE,OAAO,GAAG,MAAM,CAEpD;AAiBD,oFAAoF;AACpF,wBAAsB,SAAS,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,CAG7D;AAED;;;;GAIG;AACH,wBAAgB,eAAe,CAAC,GAAG,GAAE,MAAmB,GAAG,MAAM,CAKhE;AAED;;;;;;;;;;GAUG;AACH,eAAO,MAAM,UAAU,GAAI,OAAO,MAAM,EAAE,QAAQ,MAAM,EAAE,QAAQ,MAAM,EAAE,SAAS,MAAM,KAAG,MAC3C,CAAC;AAElD;;;GAGG;AACH,eAAO,MAAM,uBAAuB,GAAI,OAAO,MAAM,KAAG,MAA0B,CAAC;AAEnF,mEAAmE;AACnE,eAAO,MAAM,OAAO,GAAI,SAAS,MAAM,KAAG,MAA2C,CAAC"}
|
package/dist/layout.js
CHANGED
|
@@ -13,9 +13,6 @@
|
|
|
13
13
|
* workflows/<wf>/experiments/<expId>/runs/<caseId>.<repeat>.json RunRecord (write-once)
|
|
14
14
|
* workflows/<wf>/experiments/<expId>/attempts/a<n>.json attempt claim (write-once)
|
|
15
15
|
*
|
|
16
|
-
* The v1 layout (`evals/datasets/<dataset>/…`, experiments nested in their
|
|
17
|
-
* dataset) is read only by `migrateLegacyLayout`.
|
|
18
|
-
*
|
|
19
16
|
* WRITE-ONCE REVISIONS, NOT ONE FILE. Editing a case writes `r<n+1>`; nothing
|
|
20
17
|
* is rewritten, so a version that pinned `r3` by sha256 keeps loading exactly
|
|
21
18
|
* what it pinned however often the case is edited. Revision and version numbers
|
|
@@ -28,10 +25,6 @@
|
|
|
28
25
|
* to load. A marker beside it leaves history intact.
|
|
29
26
|
*/
|
|
30
27
|
export const WORKFLOWS_PREFIX = "workflows/";
|
|
31
|
-
/** The v1 layout's root; only `migrateLegacyLayout` reads it. */
|
|
32
|
-
export const LEGACY_EVALS_PREFIX = "evals/datasets/";
|
|
33
|
-
/** Written once `migrateLegacyLayout` has copied every legacy dataset. */
|
|
34
|
-
export const MIGRATION_MARKER_KEY = "evals/MIGRATED.json";
|
|
35
28
|
const WORKFLOW_ID_RE = /^[A-Za-z0-9._-]{1,128}$/;
|
|
36
29
|
const DATASET_RE = /^[a-z0-9][a-z0-9-]{0,62}$/;
|
|
37
30
|
const CASE_ID_RE = /^[A-Za-z0-9._-]{1,80}$/;
|
package/dist/layout.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"layout.js","sourceRoot":"","sources":["../src/layout.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"layout.js","sourceRoot":"","sources":["../src/layout.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AAEH,MAAM,CAAC,MAAM,gBAAgB,GAAG,YAAY,CAAC;AAE7C,MAAM,cAAc,GAAG,yBAAyB,CAAC;AACjD,MAAM,UAAU,GAAG,2BAA2B,CAAC;AAC/C,MAAM,UAAU,GAAG,wBAAwB,CAAC;AAC5C,MAAM,SAAS,GAAG,6BAA6B,CAAC;AAEhD,6EAA6E;AAC7E,MAAM,UAAU,YAAY,CAAC,EAAU;IACrC,OAAO,cAAc,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,EAAE,KAAK,GAAG,IAAI,EAAE,KAAK,IAAI,CAAC;AAC9D,CAAC;AAED,MAAM,UAAU,aAAa,CAAC,IAAY;IACxC,OAAO,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC/B,CAAC;AAED,MAAM,UAAU,QAAQ,CAAC,EAAU;IACjC,OAAO,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,EAAE,KAAK,GAAG,IAAI,EAAE,KAAK,IAAI,CAAC;AAC1D,CAAC;AAED,MAAM,UAAU,cAAc,CAAC,EAAU;IACvC,OAAO,SAAS,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;AAC5B,CAAC;AAED,MAAM,UAAU,gBAAgB,CAAC,EAAU;IACzC,IAAI,CAAC,YAAY,CAAC,EAAE,CAAC;QAAE,MAAM,IAAI,KAAK,CAAC,gBAAgB,EAAE,gBAAgB,cAAc,+BAA+B,CAAC,CAAC;AAC1H,CAAC;AAED,MAAM,UAAU,iBAAiB,CAAC,IAAY;IAC5C,IAAI,CAAC,aAAa,CAAC,IAAI,CAAC,EAAE,CAAC;QACzB,MAAM,IAAI,KAAK,CAAC,iBAAiB,IAAI,gBAAgB,UAAU,+BAA+B,CAAC,CAAC;IAClG,CAAC;AACH,CAAC;AAED,MAAM,UAAU,YAAY,CAAC,EAAU;IACrC,IAAI,CAAC,QAAQ,CAAC,EAAE,CAAC;QAAE,MAAM,IAAI,KAAK,CAAC,YAAY,EAAE,gBAAgB,UAAU,EAAE,CAAC,CAAC;AACjF,CAAC;AAED,MAAM,UAAU,kBAAkB,CAAC,EAAU;IAC3C,IAAI,CAAC,cAAc,CAAC,EAAE,CAAC;QAAE,MAAM,IAAI,KAAK,CAAC,kBAAkB,EAAE,4CAA4C,CAAC,CAAC;AAC7G,CAAC;AAED,MAAM,CAAC,MAAM,cAAc,GAAG,CAAC,UAAkB,EAAU,EAAE,CAAC,GAAG,gBAAgB,GAAG,UAAU,GAAG,CAAC;AAClG,MAAM,CAAC,MAAM,cAAc,GAAG,CAAC,UAAkB,EAAU,EAAE,CAAC,GAAG,cAAc,CAAC,UAAU,CAAC,WAAW,CAAC;AACvG,MAAM,CAAC,MAAM,aAAa,GAAG,CAAC,UAAkB,EAAE,OAAe,EAAU,EAAE,CAAC,GAAG,cAAc,CAAC,UAAU,CAAC,GAAG,OAAO,GAAG,CAAC;AACzH,MAAM,CAAC,MAAM,SAAS,GAAG,CAAC,UAAkB,EAAE,OAAe,EAAU,EAAE,CAAC,GAAG,aAAa,CAAC,UAAU,EAAE,OAAO,CAAC,aAAa,CAAC;AAC7H,MAAM,CAAC,MAAM,WAAW,GAAG,CAAC,UAAkB,EAAE,OAAe,EAAU,EAAE,CAAC,GAAG,aAAa,CAAC,UAAU,EAAE,OAAO,CAAC,QAAQ,CAAC;AAC1H,MAAM,CAAC,MAAM,aAAa,GAAG,CAAC,UAAkB,EAAE,OAAe,EAAE,MAAc,EAAU,EAAE,CAC3F,GAAG,WAAW,CAAC,UAAU,EAAE,OAAO,CAAC,GAAG,MAAM,GAAG,CAAC;AAClD,MAAM,CAAC,MAAM,UAAU,GAAG,CAAC,UAAkB,EAAE,OAAe,EAAE,MAAc,EAAE,GAAW,EAAU,EAAE,CACrG,GAAG,aAAa,CAAC,UAAU,EAAE,OAAO,EAAE,MAAM,CAAC,IAAI,GAAG,OAAO,CAAC;AAC9D,MAAM,CAAC,MAAM,YAAY,GAAG,CAAC,UAAkB,EAAE,OAAe,EAAE,MAAc,EAAU,EAAE,CAC1F,GAAG,aAAa,CAAC,UAAU,EAAE,OAAO,EAAE,MAAM,CAAC,cAAc,CAAC;AAC9D,MAAM,CAAC,MAAM,cAAc,GAAG,CAAC,UAAkB,EAAE,OAAe,EAAU,EAAE,CAAC,GAAG,aAAa,CAAC,UAAU,EAAE,OAAO,CAAC,WAAW,CAAC;AAChI,MAAM,CAAC,MAAM,UAAU,GAAG,CAAC,UAAkB,EAAE,OAAe,EAAE,MAAc,EAAU,EAAE,CACxF,GAAG,cAAc,CAAC,UAAU,EAAE,OAAO,CAAC,GAAG,MAAM,OAAO,CAAC;AACzD,MAAM,CAAC,MAAM,cAAc,GAAG,CAAC,UAAkB,EAAE,OAAe,EAAU,EAAE,CAAC,GAAG,aAAa,CAAC,UAAU,EAAE,OAAO,CAAC,WAAW,CAAC;AAChI,MAAM,CAAC,MAAM,UAAU,GAAG,CAAC,UAAkB,EAAE,OAAe,EAAE,OAAe,EAAU,EAAE,CACzF,GAAG,cAAc,CAAC,UAAU,EAAE,OAAO,CAAC,IAAI,OAAO,OAAO,CAAC;AAC3D,MAAM,CAAC,MAAM,iBAAiB,GAAG,CAAC,UAAkB,EAAU,EAAE,CAAC,GAAG,cAAc,CAAC,UAAU,CAAC,cAAc,CAAC;AAC7G,MAAM,CAAC,MAAM,gBAAgB,GAAG,CAAC,UAAkB,EAAE,KAAa,EAAU,EAAE,CAAC,GAAG,iBAAiB,CAAC,UAAU,CAAC,GAAG,KAAK,GAAG,CAAC;AAC3H,MAAM,CAAC,MAAM,iBAAiB,GAAG,CAAC,UAAkB,EAAE,KAAa,EAAU,EAAE,CAC7E,GAAG,gBAAgB,CAAC,UAAU,EAAE,KAAK,CAAC,WAAW,CAAC;AACpD,MAAM,CAAC,MAAM,UAAU,GAAG,CAAC,UAAkB,EAAE,KAAa,EAAU,EAAE,CAAC,GAAG,gBAAgB,CAAC,UAAU,EAAE,KAAK,CAAC,OAAO,CAAC;AACvH,MAAM,CAAC,MAAM,MAAM,GAAG,CAAC,UAAkB,EAAE,KAAa,EAAE,MAAc,EAAE,MAAc,EAAU,EAAE,CAClG,GAAG,UAAU,CAAC,UAAU,EAAE,KAAK,CAAC,GAAG,MAAM,IAAI,MAAM,OAAO,CAAC;AAC7D,MAAM,CAAC,MAAM,cAAc,GAAG,CAAC,UAAkB,EAAE,KAAa,EAAU,EAAE,CAAC,GAAG,gBAAgB,CAAC,UAAU,EAAE,KAAK,CAAC,WAAW,CAAC;AAC/H,MAAM,CAAC,MAAM,UAAU,GAAG,CAAC,UAAkB,EAAE,KAAa,EAAE,OAAe,EAAU,EAAE,CACvF,GAAG,cAAc,CAAC,UAAU,EAAE,KAAK,CAAC,IAAI,OAAO,OAAO,CAAC;AAEzD,+EAA+E;AAC/E,MAAM,UAAU,cAAc,CAAC,UAAkB,EAAE,KAAa,EAAE,GAAW;IAC3E,MAAM,MAAM,GAAG,cAAc,CAAC,UAAU,EAAE,KAAK,CAAC,CAAC;IACjD,IAAI,CAAC,GAAG,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC;QAAE,OAAO,IAAI,CAAC;IACnE,MAAM,CAAC,GAAG,eAAe,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;IAC1E,OAAO,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAE,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;AAC/C,CAAC;AAED,0FAA0F;AAC1F,MAAM,UAAU,UAAU,CAAC,UAAkB,EAAE,OAAe,EAAE,MAAc,EAAE,GAAW;IACzF,MAAM,MAAM,GAAG,aAAa,CAAC,UAAU,EAAE,OAAO,EAAE,MAAM,CAAC,CAAC;IAC1D,IAAI,CAAC,GAAG,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC;QAAE,OAAO,IAAI,CAAC;IACnE,MAAM,CAAC,GAAG,eAAe,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;IAC1E,OAAO,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAE,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;AAC/C,CAAC;AAED,mEAAmE;AACnE,MAAM,UAAU,aAAa,CAAC,UAAkB,EAAE,OAAe,EAAE,GAAW;IAC5E,MAAM,MAAM,GAAG,WAAW,CAAC,UAAU,EAAE,OAAO,CAAC,CAAC;IAChD,IAAI,CAAC,GAAG,CAAC,UAAU,CAAC,MAAM,CAAC;QAAE,OAAO,IAAI,CAAC;IACzC,MAAM,IAAI,GAAG,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;IACtC,MAAM,KAAK,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC;IAChC,IAAI,KAAK,GAAG,CAAC;QAAE,OAAO,IAAI,CAAC;IAC3B,MAAM,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC;IAChC,OAAO,QAAQ,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;AAClC,CAAC;AAED,MAAM,UAAU,iBAAiB,CAAC,UAAkB,EAAE,OAAe,EAAE,GAAW;IAChF,MAAM,MAAM,GAAG,cAAc,CAAC,UAAU,EAAE,OAAO,CAAC,CAAC;IACnD,IAAI,CAAC,GAAG,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC;QAAE,OAAO,IAAI,CAAC;IACnE,MAAM,EAAE,GAAG,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC;IACrD,OAAO,QAAQ,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;AAClC,CAAC;AAED,MAAM,UAAU,cAAc,CAAC,UAAkB,EAAE,OAAe,EAAE,GAAW;IAC7E,MAAM,MAAM,GAAG,cAAc,CAAC,UAAU,EAAE,OAAO,CAAC,CAAC;IACnD,IAAI,CAAC,GAAG,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC;QAAE,OAAO,IAAI,CAAC;IACnE,MAAM,CAAC,GAAG,eAAe,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;IAC1E,OAAO,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAE,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;AAC/C,CAAC;AAED,MAAM,UAAU,uBAAuB,CAAC,UAAkB,EAAE,GAAW;IACrE,MAAM,MAAM,GAAG,iBAAiB,CAAC,UAAU,CAAC,CAAC;IAC7C,IAAI,CAAC,GAAG,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,YAAY,CAAC;QAAE,OAAO,IAAI,CAAC;IACxE,MAAM,EAAE,GAAG,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,YAAY,CAAC,MAAM,CAAC,CAAC;IAC1D,OAAO,cAAc,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;AACxC,CAAC;AAED,yDAAyD;AACzD,MAAM,UAAU,UAAU,CAAC,UAAkB,EAAE,KAAa,EAAE,GAAW;IACvE,MAAM,MAAM,GAAG,UAAU,CAAC,UAAU,EAAE,KAAK,CAAC,CAAC;IAC7C,IAAI,CAAC,GAAG,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC;QAAE,OAAO,IAAI,CAAC;IACnE,MAAM,IAAI,GAAG,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC;IACvD,MAAM,GAAG,GAAG,IAAI,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC;IAClC,IAAI,GAAG,IAAI,CAAC;QAAE,OAAO,IAAI,CAAC;IAC1B,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;IAClC,MAAM,MAAM,GAAG,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC;IAC3C,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,MAAM,CAAC,IAAI,MAAM,GAAG,CAAC;QAAE,OAAO,IAAI,CAAC;IAC9E,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,CAAC;AAC5B,CAAC;AAED,2DAA2D;AAC3D,MAAM,UAAU,OAAO,CAAC,IAAc,EAAE,KAAqC;IAC3E,IAAI,GAAG,GAAG,CAAC,CAAC;IACZ,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;QACvB,MAAM,CAAC,GAAG,KAAK,CAAC,GAAG,CAAC,CAAC;QACrB,IAAI,CAAC,KAAK,IAAI,IAAI,CAAC,GAAG,GAAG;YAAE,GAAG,GAAG,CAAC,CAAC;IACrC,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,aAAa,CAAC,KAAc;IAC1C,OAAO,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,KAAK,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,GAAG,IAAI,CAAC;AACzD,CAAC;AAED,SAAS,QAAQ,CAAC,KAAc;IAC9B,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC;QAAE,OAAO,KAAK,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;IACrD,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,KAAK,IAAI,EAAE,CAAC;QAChD,4EAA4E;QAC5E,6CAA6C;QAC7C,MAAM,GAAG,GAAG,MAAM,CAAC,MAAM,CAAC,IAAI,CAA4B,CAAC;QAC3D,KAAK,MAAM,CAAC,IAAI,MAAM,CAAC,IAAI,CAAC,KAAgC,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC;YACrE,MAAM,CAAC,GAAI,KAAiC,CAAC,CAAC,CAAC,CAAC;YAChD,IAAI,CAAC,KAAK,SAAS;gBAAE,GAAG,CAAC,CAAC,CAAC,GAAG,QAAQ,CAAC,CAAC,CAAC,CAAC;QAC5C,CAAC;QACD,OAAO,GAAG,CAAC;IACb,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,oFAAoF;AACpF,MAAM,CAAC,KAAK,UAAU,SAAS,CAAC,IAAY;IAC1C,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,SAAS,EAAE,IAAI,WAAW,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC;IACrF,OAAO,CAAC,GAAG,IAAI,UAAU,CAAC,MAAM,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;AAC1F,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,eAAe,CAAC,MAAc,IAAI,CAAC,GAAG,EAAE;IACtD,MAAM,QAAQ,GAAG,sCAAsC,CAAC;IACxD,IAAI,IAAI,GAAG,EAAE,CAAC;IACd,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,EAAE,CAAC,EAAE;QAAE,IAAI,IAAI,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,EAAE,GAAG,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC;IAC1F,OAAO,OAAO,GAAG,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC,QAAQ,CAAC,CAAC,EAAE,GAAG,CAAC,IAAI,IAAI,EAAE,CAAC;AAC5D,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,CAAC,MAAM,UAAU,GAAG,CAAC,KAAa,EAAE,MAAc,EAAE,MAAc,EAAE,OAAe,EAAU,EAAE,CACnG,QAAQ,KAAK,IAAI,MAAM,IAAI,MAAM,KAAK,OAAO,EAAE,CAAC;AAElD;;;GAGG;AACH,MAAM,CAAC,MAAM,uBAAuB,GAAG,CAAC,KAAa,EAAU,EAAE,CAAC,QAAQ,KAAK,GAAG,CAAC;AAEnF,mEAAmE;AACnE,MAAM,CAAC,MAAM,OAAO,GAAG,CAAC,OAAe,EAAU,EAAE,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC"}
|
package/dist/storage.js
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* Datasets, cases, versions and experiments over an injected store.
|
|
3
3
|
*
|
|
4
|
-
* The library does no I/O of its own: the CLI
|
|
5
|
-
*
|
|
4
|
+
* The library does no I/O of its own: the CLI and ROC pass `vaultObjectStore`
|
|
5
|
+
* over a vault scope's objects, tests `memoryStore()`. Everything is plain JSON
|
|
6
6
|
* at the keys in layout.ts.
|
|
7
7
|
*/
|
|
8
8
|
import { WORKFLOWS_PREFIX, archiveKey, archivedIdFromKey, archivedPrefix, assertCaseId, assertDatasetName, assertExperimentId, assertWorkflowId, attemptFromKey, attemptKey, attemptsPrefix, canonicalJson, caseDirPrefix, caseIdFromKey, caseRevKey, casesPrefix, datasetsPrefix, experimentMetaKey, experimentsPrefix, headerKey, highest, isDatasetName, isExperimentId, isWorkflowId, revFromKey, runFromKey, runKey, runsPrefix, sha256Hex, tombstoneKey, versionFromKey, versionKey, versionsPrefix, } from "./layout.js";
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* `EvalStore` over a vault scope's object storage.
|
|
3
|
+
*
|
|
4
|
+
* Evaluations live in the vault the workflow runs in, in the workflow's scope:
|
|
5
|
+
* `/api/v1/vaults/<vault>/scopes/<scope>/objects/evals/workflows/<wf>/…`. So
|
|
6
|
+
* the vault decides who may read and write them, the same way it decides who
|
|
7
|
+
* may run the workflow: a read grant on the scope reads datasets and
|
|
8
|
+
* experiments, a write grant manages datasets and runs experiments.
|
|
9
|
+
*
|
|
10
|
+
* Takes the objects client STRUCTURALLY — vault-sdk's `vault.scope(s).objects`
|
|
11
|
+
* fits, and so does an in-memory fake — so this package keeps no dependency on
|
|
12
|
+
* the vault SDK and stays browser-safe.
|
|
13
|
+
*/
|
|
14
|
+
import type { EvalStore } from "./storage.js";
|
|
15
|
+
/** What the store needs of a vault scope's objects client. */
|
|
16
|
+
export interface VaultObjects {
|
|
17
|
+
upload(path: string, data: Uint8Array, opts: {
|
|
18
|
+
contentType?: string;
|
|
19
|
+
ifNoneMatch?: "*";
|
|
20
|
+
}): Promise<unknown>;
|
|
21
|
+
get(path: string): Promise<Uint8Array>;
|
|
22
|
+
list(opts: {
|
|
23
|
+
prefix?: string;
|
|
24
|
+
}): Promise<{
|
|
25
|
+
items: Array<{
|
|
26
|
+
path: string;
|
|
27
|
+
}>;
|
|
28
|
+
}>;
|
|
29
|
+
}
|
|
30
|
+
export interface VaultObjectStoreOptions {
|
|
31
|
+
/** Path prefix inside the scope's objects. Default `"evals/"`. */
|
|
32
|
+
root?: string;
|
|
33
|
+
}
|
|
34
|
+
/**
|
|
35
|
+
* The server or the client dropped `If-None-Match: *`: a create-only upload of
|
|
36
|
+
* a path that exists was accepted. Writing on would turn every write-once key
|
|
37
|
+
* (case revisions, dataset versions) into an overwrite that still answers
|
|
38
|
+
* "won", so the store refuses instead.
|
|
39
|
+
*/
|
|
40
|
+
export declare class ConditionalWriteUnsupportedError extends Error {
|
|
41
|
+
constructor();
|
|
42
|
+
}
|
|
43
|
+
/** The one key the write-once probe uses, so repeated sessions do not pile up keys nobody can delete. */
|
|
44
|
+
export declare const CONDITIONAL_WRITE_PROBE_KEY = ".probe/conditional-write";
|
|
45
|
+
export declare function vaultObjectStore(objects: VaultObjects, opts?: VaultObjectStoreOptions): EvalStore;
|
|
46
|
+
//# sourceMappingURL=vault-store.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"vault-store.d.ts","sourceRoot":"","sources":["../src/vault-store.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AACH,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,cAAc,CAAC;AAE9C,8DAA8D;AAC9D,MAAM,WAAW,YAAY;IAC3B,MAAM,CACJ,IAAI,EAAE,MAAM,EACZ,IAAI,EAAE,UAAU,EAChB,IAAI,EAAE;QAAE,WAAW,CAAC,EAAE,MAAM,CAAC;QAAC,WAAW,CAAC,EAAE,GAAG,CAAA;KAAE,GAChD,OAAO,CAAC,OAAO,CAAC,CAAC;IACpB,GAAG,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,UAAU,CAAC,CAAC;IACvC,IAAI,CAAC,IAAI,EAAE;QAAE,MAAM,CAAC,EAAE,MAAM,CAAA;KAAE,GAAG,OAAO,CAAC;QAAE,KAAK,EAAE,KAAK,CAAC;YAAE,IAAI,EAAE,MAAM,CAAA;SAAE,CAAC,CAAA;KAAE,CAAC,CAAC;CAC9E;AAED,MAAM,WAAW,uBAAuB;IACtC,kEAAkE;IAClE,IAAI,CAAC,EAAE,MAAM,CAAC;CACf;AAwBD;;;;;GAKG;AACH,qBAAa,gCAAiC,SAAQ,KAAK;;CAS1D;AAED,yGAAyG;AACzG,eAAO,MAAM,2BAA2B,6BAA6B,CAAC;AAEtE,wBAAgB,gBAAgB,CAAC,OAAO,EAAE,YAAY,EAAE,IAAI,GAAE,uBAA4B,GAAG,SAAS,CA+FrG"}
|
|
@@ -0,0 +1,127 @@
|
|
|
1
|
+
const errorField = (err, field) => err && typeof err === "object" ? err[field] : undefined;
|
|
2
|
+
/**
|
|
3
|
+
* The object is not there: vault-api's `OBJECT_NOT_FOUND`, by its code. A bare
|
|
4
|
+
* 404 is not enough — a wrong vault-api URL, an unmounted route or a proxy
|
|
5
|
+
* answers one too, and reading that as "absent" would let `putCase`'s
|
|
6
|
+
* write-once check pass on a read that never happened.
|
|
7
|
+
*/
|
|
8
|
+
const isNotFound = (err) => errorField(err, "code") === "OBJECT_NOT_FOUND";
|
|
9
|
+
/**
|
|
10
|
+
* A create-only upload lost: vault-api answers `If-None-Match: *` on a taken
|
|
11
|
+
* path with 409 `OBJECT_EXISTS`. Nothing else counts — any other refusal is a
|
|
12
|
+
* failure, not someone else having won.
|
|
13
|
+
*/
|
|
14
|
+
const isTaken = (err) => errorField(err, "status") === 409 && errorField(err, "code") === "OBJECT_EXISTS";
|
|
15
|
+
const encoder = new TextEncoder();
|
|
16
|
+
const decoder = new TextDecoder();
|
|
17
|
+
/**
|
|
18
|
+
* The server or the client dropped `If-None-Match: *`: a create-only upload of
|
|
19
|
+
* a path that exists was accepted. Writing on would turn every write-once key
|
|
20
|
+
* (case revisions, dataset versions) into an overwrite that still answers
|
|
21
|
+
* "won", so the store refuses instead.
|
|
22
|
+
*/
|
|
23
|
+
export class ConditionalWriteUnsupportedError extends Error {
|
|
24
|
+
constructor() {
|
|
25
|
+
super("This vault does not honour create-only uploads (If-None-Match: *): an upload of a path that already " +
|
|
26
|
+
"exists was accepted. Evaluations need vault-api with Cerebellum-Network/ddc-node#1375 and " +
|
|
27
|
+
"@cef-ai/vault-sdk >= 5.8.0; nothing was written.");
|
|
28
|
+
this.name = "ConditionalWriteUnsupportedError";
|
|
29
|
+
}
|
|
30
|
+
}
|
|
31
|
+
/** The one key the write-once probe uses, so repeated sessions do not pile up keys nobody can delete. */
|
|
32
|
+
export const CONDITIONAL_WRITE_PROBE_KEY = ".probe/conditional-write";
|
|
33
|
+
export function vaultObjectStore(objects, opts = {}) {
|
|
34
|
+
const root = (opts.root ?? "evals/").replace(/\/*$/, "/").replace(/^\/+/, "");
|
|
35
|
+
const full = (key) => `${root}${key}`;
|
|
36
|
+
const createOnly = async (path, body) => {
|
|
37
|
+
try {
|
|
38
|
+
await objects.upload(path, encoder.encode(body), { contentType: "application/json", ifNoneMatch: "*" });
|
|
39
|
+
return true;
|
|
40
|
+
}
|
|
41
|
+
catch (err) {
|
|
42
|
+
if (isTaken(err))
|
|
43
|
+
return false;
|
|
44
|
+
throw err;
|
|
45
|
+
}
|
|
46
|
+
};
|
|
47
|
+
/*
|
|
48
|
+
* ONCE PER STORE, BEFORE THE FIRST WRITE-ONCE WRITE: prove the create-only
|
|
49
|
+
* upload is real. Two create-only uploads of the same fixed key; the second
|
|
50
|
+
* must lose. An old vault-sdk (which drops the option) or a vault-api that
|
|
51
|
+
* ignores the header accepts it, and then `putIfAbsent` would overwrite
|
|
52
|
+
* silently while answering `true`. Two round trips per store; a reader never
|
|
53
|
+
* pays them. A probe that failed for any other reason (network, no grant) is
|
|
54
|
+
* not remembered, so the next write asks again.
|
|
55
|
+
*/
|
|
56
|
+
let probe = null;
|
|
57
|
+
const proveCreateOnly = () => {
|
|
58
|
+
probe ??= (async () => {
|
|
59
|
+
const key = full(CONDITIONAL_WRITE_PROBE_KEY);
|
|
60
|
+
await createOnly(key, "{}");
|
|
61
|
+
if (await createOnly(key, "{}"))
|
|
62
|
+
throw new ConditionalWriteUnsupportedError();
|
|
63
|
+
})().catch((err) => {
|
|
64
|
+
if (!(err instanceof ConditionalWriteUnsupportedError))
|
|
65
|
+
probe = null;
|
|
66
|
+
throw err;
|
|
67
|
+
});
|
|
68
|
+
return probe;
|
|
69
|
+
};
|
|
70
|
+
const list = async (prefix) => {
|
|
71
|
+
// vault-api lists every page server-side and answers once, so there is
|
|
72
|
+
// no continuation to follow and nothing to be silently cut short. If its
|
|
73
|
+
// List handler ever starts paging (a cursor in the response), this must
|
|
74
|
+
// follow it in the same change, or listings truncate without a word.
|
|
75
|
+
const res = await objects.list({ prefix: full(prefix) });
|
|
76
|
+
return res.items
|
|
77
|
+
.map((i) => i.path)
|
|
78
|
+
.filter((p) => p.startsWith(root))
|
|
79
|
+
.map((p) => p.slice(root.length))
|
|
80
|
+
// The probe's key is the store's own, not eval data.
|
|
81
|
+
.filter((k) => k.startsWith(prefix) && k !== CONDITIONAL_WRITE_PROBE_KEY)
|
|
82
|
+
.sort();
|
|
83
|
+
};
|
|
84
|
+
return {
|
|
85
|
+
list,
|
|
86
|
+
/** Derived from the flat listing: the object API has no delimiter. */
|
|
87
|
+
async listPrefixes(prefix) {
|
|
88
|
+
const out = new Set();
|
|
89
|
+
for (const key of await list(prefix)) {
|
|
90
|
+
const slash = key.indexOf("/", prefix.length);
|
|
91
|
+
if (slash > prefix.length)
|
|
92
|
+
out.add(key.slice(0, slash + 1));
|
|
93
|
+
}
|
|
94
|
+
return [...out].sort();
|
|
95
|
+
},
|
|
96
|
+
async get(key) {
|
|
97
|
+
try {
|
|
98
|
+
return decoder.decode(await objects.get(full(key)));
|
|
99
|
+
}
|
|
100
|
+
catch (err) {
|
|
101
|
+
// Only a missing object is `null`. Anything else (no grant, vault-api
|
|
102
|
+
// down) must throw, or `putCase`'s write-once check would pass on a
|
|
103
|
+
// failed read and overwrite a real case.
|
|
104
|
+
if (isNotFound(err))
|
|
105
|
+
return null;
|
|
106
|
+
throw err;
|
|
107
|
+
}
|
|
108
|
+
},
|
|
109
|
+
async put(key, body) {
|
|
110
|
+
await objects.upload(full(key), encoder.encode(body), { contentType: "application/json" });
|
|
111
|
+
},
|
|
112
|
+
/**
|
|
113
|
+
* The server decides: `If-None-Match: *` on the upload, and a taken path
|
|
114
|
+
* answers `false`. No read first and no unconditional fallback: either
|
|
115
|
+
* could let two writers both believe they cut the same version. The first
|
|
116
|
+
* call proves the mechanism works (see `proveCreateOnly`) and refuses with
|
|
117
|
+
* `ConditionalWriteUnsupportedError` when it does not.
|
|
118
|
+
*/
|
|
119
|
+
async putIfAbsent(key, body) {
|
|
120
|
+
await proveCreateOnly();
|
|
121
|
+
return createOnly(full(key), body);
|
|
122
|
+
},
|
|
123
|
+
/** Pinned by `vault-store.test.ts`, which reads `ifNoneMatch` off the upload this store sent. */
|
|
124
|
+
atomicity: "conditional-write",
|
|
125
|
+
};
|
|
126
|
+
}
|
|
127
|
+
//# sourceMappingURL=vault-store.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"vault-store.js","sourceRoot":"","sources":["../src/vault-store.ts"],"names":[],"mappings":"AA+BA,MAAM,UAAU,GAAG,CAAC,GAAY,EAAE,KAAwB,EAAW,EAAE,CACrE,GAAG,IAAI,OAAO,GAAG,KAAK,QAAQ,CAAC,CAAC,CAAE,GAA+B,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;AAEvF;;;;;GAKG;AACH,MAAM,UAAU,GAAG,CAAC,GAAY,EAAW,EAAE,CAAC,UAAU,CAAC,GAAG,EAAE,MAAM,CAAC,KAAK,kBAAkB,CAAC;AAE7F;;;;GAIG;AACH,MAAM,OAAO,GAAG,CAAC,GAAY,EAAW,EAAE,CACxC,UAAU,CAAC,GAAG,EAAE,QAAQ,CAAC,KAAK,GAAG,IAAI,UAAU,CAAC,GAAG,EAAE,MAAM,CAAC,KAAK,eAAe,CAAC;AAEnF,MAAM,OAAO,GAAG,IAAI,WAAW,EAAE,CAAC;AAClC,MAAM,OAAO,GAAG,IAAI,WAAW,EAAE,CAAC;AAElC;;;;;GAKG;AACH,MAAM,OAAO,gCAAiC,SAAQ,KAAK;IACzD;QACE,KAAK,CACH,sGAAsG;YACpG,4FAA4F;YAC5F,kDAAkD,CACrD,CAAC;QACF,IAAI,CAAC,IAAI,GAAG,kCAAkC,CAAC;IACjD,CAAC;CACF;AAED,yGAAyG;AACzG,MAAM,CAAC,MAAM,2BAA2B,GAAG,0BAA0B,CAAC;AAEtE,MAAM,UAAU,gBAAgB,CAAC,OAAqB,EAAE,OAAgC,EAAE;IACxF,MAAM,IAAI,GAAG,CAAC,IAAI,CAAC,IAAI,IAAI,QAAQ,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;IAC9E,MAAM,IAAI,GAAG,CAAC,GAAW,EAAE,EAAE,CAAC,GAAG,IAAI,GAAG,GAAG,EAAE,CAAC;IAE9C,MAAM,UAAU,GAAG,KAAK,EAAE,IAAY,EAAE,IAAY,EAAoB,EAAE;QACxE,IAAI,CAAC;YACH,MAAM,OAAO,CAAC,MAAM,CAAC,IAAI,EAAE,OAAO,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,EAAE,WAAW,EAAE,kBAAkB,EAAE,WAAW,EAAE,GAAG,EAAE,CAAC,CAAC;YACxG,OAAO,IAAI,CAAC;QACd,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,IAAI,OAAO,CAAC,GAAG,CAAC;gBAAE,OAAO,KAAK,CAAC;YAC/B,MAAM,GAAG,CAAC;QACZ,CAAC;IACH,CAAC,CAAC;IAEF;;;;;;;;OAQG;IACH,IAAI,KAAK,GAAyB,IAAI,CAAC;IACvC,MAAM,eAAe,GAAG,GAAkB,EAAE;QAC1C,KAAK,KAAK,CAAC,KAAK,IAAI,EAAE;YACpB,MAAM,GAAG,GAAG,IAAI,CAAC,2BAA2B,CAAC,CAAC;YAC9C,MAAM,UAAU,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC;YAC5B,IAAI,MAAM,UAAU,CAAC,GAAG,EAAE,IAAI,CAAC;gBAAE,MAAM,IAAI,gCAAgC,EAAE,CAAC;QAChF,CAAC,CAAC,EAAE,CAAC,KAAK,CAAC,CAAC,GAAY,EAAE,EAAE;YAC1B,IAAI,CAAC,CAAC,GAAG,YAAY,gCAAgC,CAAC;gBAAE,KAAK,GAAG,IAAI,CAAC;YACrE,MAAM,GAAG,CAAC;QACZ,CAAC,CAAC,CAAC;QACH,OAAO,KAAK,CAAC;IACf,CAAC,CAAC;IAEF,MAAM,IAAI,GAAG,KAAK,EAAE,MAAc,EAAqB,EAAE;QACvD,uEAAuE;QACvE,yEAAyE;QACzE,wEAAwE;QACxE,qEAAqE;QACrE,MAAM,GAAG,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;QACzD,OAAO,GAAG,CAAC,KAAK;aACb,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;aAClB,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC;aACjC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;YACjC,qDAAqD;aACpD,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,UAAU,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,2BAA2B,CAAC;aACxE,IAAI,EAAE,CAAC;IACZ,CAAC,CAAC;IAEF,OAAO;QACL,IAAI;QAEJ,sEAAsE;QACtE,KAAK,CAAC,YAAY,CAAC,MAAM;YACvB,MAAM,GAAG,GAAG,IAAI,GAAG,EAAU,CAAC;YAC9B,KAAK,MAAM,GAAG,IAAI,MAAM,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC;gBACrC,MAAM,KAAK,GAAG,GAAG,CAAC,OAAO,CAAC,GAAG,EAAE,MAAM,CAAC,MAAM,CAAC,CAAC;gBAC9C,IAAI,KAAK,GAAG,MAAM,CAAC,MAAM;oBAAE,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,KAAK,GAAG,CAAC,CAAC,CAAC,CAAC;YAC9D,CAAC;YACD,OAAO,CAAC,GAAG,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC;QACzB,CAAC;QAED,KAAK,CAAC,GAAG,CAAC,GAAG;YACX,IAAI,CAAC;gBACH,OAAO,OAAO,CAAC,MAAM,CAAC,MAAM,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;YACtD,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACb,sEAAsE;gBACtE,oEAAoE;gBACpE,yCAAyC;gBACzC,IAAI,UAAU,CAAC,GAAG,CAAC;oBAAE,OAAO,IAAI,CAAC;gBACjC,MAAM,GAAG,CAAC;YACZ,CAAC;QACH,CAAC;QAED,KAAK,CAAC,GAAG,CAAC,GAAG,EAAE,IAAI;YACjB,MAAM,OAAO,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,OAAO,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,EAAE,WAAW,EAAE,kBAAkB,EAAE,CAAC,CAAC;QAC7F,CAAC;QAED;;;;;;WAMG;QACH,KAAK,CAAC,WAAW,CAAC,GAAG,EAAE,IAAI;YACzB,MAAM,eAAe,EAAE,CAAC;YACxB,OAAO,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,IAAI,CAAC,CAAC;QACrC,CAAC;QAED,iGAAiG;QACjG,SAAS,EAAE,mBAAmB;KAC/B,CAAC;AACJ,CAAC"}
|
package/package.json
CHANGED
package/dist/migrate.d.ts
DELETED
|
@@ -1,30 +0,0 @@
|
|
|
1
|
-
import { type EvalStore } from "./storage.js";
|
|
2
|
-
export interface MigrationReport {
|
|
3
|
-
/** The marker was already there; nothing was read or written. */
|
|
4
|
-
alreadyMigrated: boolean;
|
|
5
|
-
datasets: Array<{
|
|
6
|
-
dataset: string;
|
|
7
|
-
workflowId: string;
|
|
8
|
-
keys: number;
|
|
9
|
-
experiments: number;
|
|
10
|
-
}>;
|
|
11
|
-
/** Keys this call wrote. */
|
|
12
|
-
copied: number;
|
|
13
|
-
/** Keys already at their target (an earlier run, or a v2 writer); left as they were. */
|
|
14
|
-
existing: number;
|
|
15
|
-
/** Legacy datasets not copied, and why. Any entry keeps the marker unwritten. */
|
|
16
|
-
skipped: Array<{
|
|
17
|
-
dataset: string;
|
|
18
|
-
reason: string;
|
|
19
|
-
}>;
|
|
20
|
-
}
|
|
21
|
-
/** Whether this store has v1 keys and no marker saying they were migrated. */
|
|
22
|
-
export declare function needsLegacyMigration(store: EvalStore): Promise<boolean>;
|
|
23
|
-
/**
|
|
24
|
-
* Copy every v1 dataset — header, cases, flags, versions and its experiments —
|
|
25
|
-
* into `workflows/<header.workflowId>/…`. Idempotent; see the file comment.
|
|
26
|
-
*/
|
|
27
|
-
export declare function migrateLegacyLayout(store: EvalStore, opts?: {
|
|
28
|
-
now?: Date;
|
|
29
|
-
}): Promise<MigrationReport>;
|
|
30
|
-
//# sourceMappingURL=migrate.d.ts.map
|
package/dist/migrate.d.ts.map
DELETED
|
@@ -1 +0,0 @@
|
|
|
1
|
-
{"version":3,"file":"migrate.d.ts","sourceRoot":"","sources":["../src/migrate.ts"],"names":[],"mappings":"AAyBA,OAAO,EAAiB,KAAK,SAAS,EAAE,MAAM,cAAc,CAAC;AAE7D,MAAM,WAAW,eAAe;IAC9B,iEAAiE;IACjE,eAAe,EAAE,OAAO,CAAC;IACzB,QAAQ,EAAE,KAAK,CAAC;QAAE,OAAO,EAAE,MAAM,CAAC;QAAC,UAAU,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAC;QAAC,WAAW,EAAE,MAAM,CAAA;KAAE,CAAC,CAAC;IAC5F,4BAA4B;IAC5B,MAAM,EAAE,MAAM,CAAC;IACf,wFAAwF;IACxF,QAAQ,EAAE,MAAM,CAAC;IACjB,iFAAiF;IACjF,OAAO,EAAE,KAAK,CAAC;QAAE,OAAO,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC,CAAC;CACrD;AAKD,8EAA8E;AAC9E,wBAAsB,oBAAoB,CAAC,KAAK,EAAE,SAAS,GAAG,OAAO,CAAC,OAAO,CAAC,CAG7E;AAYD;;;GAGG;AACH,wBAAsB,mBAAmB,CAAC,KAAK,EAAE,SAAS,EAAE,IAAI,GAAE;IAAE,GAAG,CAAC,EAAE,IAAI,CAAA;CAAO,GAAG,OAAO,CAAC,eAAe,CAAC,CAkE/G"}
|
package/dist/migrate.js
DELETED
|
@@ -1,113 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Moving v1 storage (`evals/datasets/<dataset>/…`, experiments nested in their
|
|
3
|
-
* dataset) to the v2 layout grouped by workflow (see layout.ts).
|
|
4
|
-
*
|
|
5
|
-
* COPY, NEVER MOVE. Every legacy key is copied byte for byte with
|
|
6
|
-
* `putIfAbsent` and left where it was: the store has no delete, a half-done
|
|
7
|
-
* migration must be safe to rerun, and an older ROC or CLI still reading the
|
|
8
|
-
* legacy keys keeps working until it is upgraded. A key already at its target
|
|
9
|
-
* is counted, not overwritten — whatever a v2 writer put there since wins.
|
|
10
|
-
*
|
|
11
|
-
* THE MARKER is written last, and only when every legacy dataset was copied.
|
|
12
|
-
* A dataset whose header cannot be read is reported and leaves the marker
|
|
13
|
-
* unwritten, so the next run tries again.
|
|
14
|
-
*/
|
|
15
|
-
import { LEGACY_EVALS_PREFIX, MIGRATION_MARKER_KEY, canonicalJson, datasetPrefix, experimentsPrefix, headerKey, isDatasetName, isWorkflowId, } from "./layout.js";
|
|
16
|
-
import { isObject, parseHeader } from "./contract.js";
|
|
17
|
-
import { childPrefixes } from "./storage.js";
|
|
18
|
-
/** The baseline ROC kept beside a v1 header before `Header.baselineId` existed. */
|
|
19
|
-
const LEGACY_BASELINE_FILE = "baseline.json";
|
|
20
|
-
/** Whether this store has v1 keys and no marker saying they were migrated. */
|
|
21
|
-
export async function needsLegacyMigration(store) {
|
|
22
|
-
if ((await store.get(MIGRATION_MARKER_KEY)) !== null)
|
|
23
|
-
return false;
|
|
24
|
-
return (await childPrefixes(store, LEGACY_EVALS_PREFIX)).length > 0;
|
|
25
|
-
}
|
|
26
|
-
function legacyBaseline(raw) {
|
|
27
|
-
if (raw === null)
|
|
28
|
-
return undefined;
|
|
29
|
-
try {
|
|
30
|
-
const v = JSON.parse(raw);
|
|
31
|
-
return isObject(v) && typeof v.baselineId === "string" && v.baselineId ? v.baselineId : undefined;
|
|
32
|
-
}
|
|
33
|
-
catch {
|
|
34
|
-
return undefined;
|
|
35
|
-
}
|
|
36
|
-
}
|
|
37
|
-
/**
|
|
38
|
-
* Copy every v1 dataset — header, cases, flags, versions and its experiments —
|
|
39
|
-
* into `workflows/<header.workflowId>/…`. Idempotent; see the file comment.
|
|
40
|
-
*/
|
|
41
|
-
export async function migrateLegacyLayout(store, opts = {}) {
|
|
42
|
-
const report = { alreadyMigrated: false, datasets: [], copied: 0, existing: 0, skipped: [] };
|
|
43
|
-
if ((await store.get(MIGRATION_MARKER_KEY)) !== null)
|
|
44
|
-
return { ...report, alreadyMigrated: true };
|
|
45
|
-
for (const prefix of await childPrefixes(store, LEGACY_EVALS_PREFIX)) {
|
|
46
|
-
const dataset = prefix.slice(LEGACY_EVALS_PREFIX.length, -1);
|
|
47
|
-
if (!isDatasetName(dataset)) {
|
|
48
|
-
// v1 enforced the same name rule, so this should not occur; if it does, it is reported, not silently dropped.
|
|
49
|
-
report.skipped.push({ dataset, reason: "the name is not a valid dataset name, so it cannot be a storage key segment" });
|
|
50
|
-
continue;
|
|
51
|
-
}
|
|
52
|
-
const rawHeader = await store.get(`${prefix}header.json`);
|
|
53
|
-
if (rawHeader === null) {
|
|
54
|
-
report.skipped.push({ dataset, reason: "no header.json" });
|
|
55
|
-
continue;
|
|
56
|
-
}
|
|
57
|
-
let header;
|
|
58
|
-
try {
|
|
59
|
-
header = parseHeader(JSON.parse(rawHeader));
|
|
60
|
-
}
|
|
61
|
-
catch (e) {
|
|
62
|
-
report.skipped.push({ dataset, reason: `header.json unreadable: ${e.message}` });
|
|
63
|
-
continue;
|
|
64
|
-
}
|
|
65
|
-
const workflowId = header.workflowId;
|
|
66
|
-
if (!isWorkflowId(workflowId)) {
|
|
67
|
-
report.skipped.push({ dataset, reason: `workflowId "${workflowId}" cannot be a storage key segment` });
|
|
68
|
-
continue;
|
|
69
|
-
}
|
|
70
|
-
const baselineId = header.baselineId ?? legacyBaseline(await store.get(`${prefix}${LEGACY_BASELINE_FILE}`));
|
|
71
|
-
const experiments = new Set();
|
|
72
|
-
let keys = 0;
|
|
73
|
-
for (const key of await store.list(prefix)) {
|
|
74
|
-
const rest = key.slice(prefix.length);
|
|
75
|
-
let target;
|
|
76
|
-
let body;
|
|
77
|
-
if (rest === LEGACY_BASELINE_FILE)
|
|
78
|
-
continue;
|
|
79
|
-
if (rest === "header.json") {
|
|
80
|
-
target = headerKey(workflowId, dataset);
|
|
81
|
-
// The header carries the ROC baseline from now on; otherwise the bytes are kept.
|
|
82
|
-
body = baselineId && !header.baselineId ? canonicalJson({ ...header, baselineId }) : rawHeader;
|
|
83
|
-
}
|
|
84
|
-
else if (rest.startsWith("experiments/")) {
|
|
85
|
-
const tail = rest.slice("experiments/".length);
|
|
86
|
-
experiments.add(tail.split("/")[0]);
|
|
87
|
-
target = `${experimentsPrefix(workflowId)}${tail}`;
|
|
88
|
-
body = await store.get(key);
|
|
89
|
-
}
|
|
90
|
-
else {
|
|
91
|
-
target = `${datasetPrefix(workflowId, dataset)}${rest}`;
|
|
92
|
-
body = await store.get(key);
|
|
93
|
-
}
|
|
94
|
-
if (body === null)
|
|
95
|
-
continue;
|
|
96
|
-
keys++;
|
|
97
|
-
if (await store.putIfAbsent(target, body))
|
|
98
|
-
report.copied++;
|
|
99
|
-
else
|
|
100
|
-
report.existing++;
|
|
101
|
-
}
|
|
102
|
-
report.datasets.push({ dataset, workflowId, keys, experiments: experiments.size });
|
|
103
|
-
}
|
|
104
|
-
if (report.skipped.length === 0) {
|
|
105
|
-
await store.put(MIGRATION_MARKER_KEY, canonicalJson({
|
|
106
|
-
migratedAt: (opts.now ?? new Date()).toISOString(),
|
|
107
|
-
datasets: report.datasets.map(({ dataset, workflowId }) => ({ dataset, workflowId })),
|
|
108
|
-
copied: report.copied,
|
|
109
|
-
}));
|
|
110
|
-
}
|
|
111
|
-
return report;
|
|
112
|
-
}
|
|
113
|
-
//# sourceMappingURL=migrate.js.map
|
package/dist/migrate.js.map
DELETED
|
@@ -1 +0,0 @@
|
|
|
1
|
-
{"version":3,"file":"migrate.js","sourceRoot":"","sources":["../src/migrate.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;GAaG;AACH,OAAO,EACL,mBAAmB,EACnB,oBAAoB,EACpB,aAAa,EACb,aAAa,EACb,iBAAiB,EACjB,SAAS,EACT,aAAa,EACb,YAAY,GACb,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,QAAQ,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AACtD,OAAO,EAAE,aAAa,EAAkB,MAAM,cAAc,CAAC;AAc7D,mFAAmF;AACnF,MAAM,oBAAoB,GAAG,eAAe,CAAC;AAE7C,8EAA8E;AAC9E,MAAM,CAAC,KAAK,UAAU,oBAAoB,CAAC,KAAgB;IACzD,IAAI,CAAC,MAAM,KAAK,CAAC,GAAG,CAAC,oBAAoB,CAAC,CAAC,KAAK,IAAI;QAAE,OAAO,KAAK,CAAC;IACnE,OAAO,CAAC,MAAM,aAAa,CAAC,KAAK,EAAE,mBAAmB,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC;AACtE,CAAC;AAED,SAAS,cAAc,CAAC,GAAkB;IACxC,IAAI,GAAG,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IACnC,IAAI,CAAC;QACH,MAAM,CAAC,GAAY,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;QACnC,OAAO,QAAQ,CAAC,CAAC,CAAC,IAAI,OAAO,CAAC,CAAC,UAAU,KAAK,QAAQ,IAAI,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC,UAAU,CAAC,CAAC,CAAC,SAAS,CAAC;IACpG,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,SAAS,CAAC;IACnB,CAAC;AACH,CAAC;AAED;;;GAGG;AACH,MAAM,CAAC,KAAK,UAAU,mBAAmB,CAAC,KAAgB,EAAE,OAAuB,EAAE;IACnF,MAAM,MAAM,GAAoB,EAAE,eAAe,EAAE,KAAK,EAAE,QAAQ,EAAE,EAAE,EAAE,MAAM,EAAE,CAAC,EAAE,QAAQ,EAAE,CAAC,EAAE,OAAO,EAAE,EAAE,EAAE,CAAC;IAC9G,IAAI,CAAC,MAAM,KAAK,CAAC,GAAG,CAAC,oBAAoB,CAAC,CAAC,KAAK,IAAI;QAAE,OAAO,EAAE,GAAG,MAAM,EAAE,eAAe,EAAE,IAAI,EAAE,CAAC;IAElG,KAAK,MAAM,MAAM,IAAI,MAAM,aAAa,CAAC,KAAK,EAAE,mBAAmB,CAAC,EAAE,CAAC;QACrE,MAAM,OAAO,GAAG,MAAM,CAAC,KAAK,CAAC,mBAAmB,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC,CAAC;QAC7D,IAAI,CAAC,aAAa,CAAC,OAAO,CAAC,EAAE,CAAC;YAC5B,8GAA8G;YAC9G,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,OAAO,EAAE,MAAM,EAAE,6EAA6E,EAAE,CAAC,CAAC;YACxH,SAAS;QACX,CAAC;QACD,MAAM,SAAS,GAAG,MAAM,KAAK,CAAC,GAAG,CAAC,GAAG,MAAM,aAAa,CAAC,CAAC;QAC1D,IAAI,SAAS,KAAK,IAAI,EAAE,CAAC;YACvB,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,OAAO,EAAE,MAAM,EAAE,gBAAgB,EAAE,CAAC,CAAC;YAC3D,SAAS;QACX,CAAC;QACD,IAAI,MAAM,CAAC;QACX,IAAI,CAAC;YACH,MAAM,GAAG,WAAW,CAAC,IAAI,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC,CAAC;QAC9C,CAAC;QAAC,OAAO,CAAC,EAAE,CAAC;YACX,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,OAAO,EAAE,MAAM,EAAE,2BAA4B,CAAW,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC;YAC5F,SAAS;QACX,CAAC;QACD,MAAM,UAAU,GAAG,MAAM,CAAC,UAAU,CAAC;QACrC,IAAI,CAAC,YAAY,CAAC,UAAU,CAAC,EAAE,CAAC;YAC9B,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,OAAO,EAAE,MAAM,EAAE,eAAe,UAAU,mCAAmC,EAAE,CAAC,CAAC;YACvG,SAAS;QACX,CAAC;QAED,MAAM,UAAU,GAAG,MAAM,CAAC,UAAU,IAAI,cAAc,CAAC,MAAM,KAAK,CAAC,GAAG,CAAC,GAAG,MAAM,GAAG,oBAAoB,EAAE,CAAC,CAAC,CAAC;QAC5G,MAAM,WAAW,GAAG,IAAI,GAAG,EAAU,CAAC;QACtC,IAAI,IAAI,GAAG,CAAC,CAAC;QACb,KAAK,MAAM,GAAG,IAAI,MAAM,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC;YAC3C,MAAM,IAAI,GAAG,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;YACtC,IAAI,MAAc,CAAC;YACnB,IAAI,IAAmB,CAAC;YACxB,IAAI,IAAI,KAAK,oBAAoB;gBAAE,SAAS;YAC5C,IAAI,IAAI,KAAK,aAAa,EAAE,CAAC;gBAC3B,MAAM,GAAG,SAAS,CAAC,UAAU,EAAE,OAAO,CAAC,CAAC;gBACxC,iFAAiF;gBACjF,IAAI,GAAG,UAAU,IAAI,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC,CAAC,aAAa,CAAC,EAAE,GAAG,MAAM,EAAE,UAAU,EAAE,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;YACjG,CAAC;iBAAM,IAAI,IAAI,CAAC,UAAU,CAAC,cAAc,CAAC,EAAE,CAAC;gBAC3C,MAAM,IAAI,GAAG,IAAI,CAAC,KAAK,CAAC,cAAc,CAAC,MAAM,CAAC,CAAC;gBAC/C,WAAW,CAAC,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAE,CAAC,CAAC;gBACrC,MAAM,GAAG,GAAG,iBAAiB,CAAC,UAAU,CAAC,GAAG,IAAI,EAAE,CAAC;gBACnD,IAAI,GAAG,MAAM,KAAK,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;YAC9B,CAAC;iBAAM,CAAC;gBACN,MAAM,GAAG,GAAG,aAAa,CAAC,UAAU,EAAE,OAAO,CAAC,GAAG,IAAI,EAAE,CAAC;gBACxD,IAAI,GAAG,MAAM,KAAK,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;YAC9B,CAAC;YACD,IAAI,IAAI,KAAK,IAAI;gBAAE,SAAS;YAC5B,IAAI,EAAE,CAAC;YACP,IAAI,MAAM,KAAK,CAAC,WAAW,CAAC,MAAM,EAAE,IAAI,CAAC;gBAAE,MAAM,CAAC,MAAM,EAAE,CAAC;;gBACtD,MAAM,CAAC,QAAQ,EAAE,CAAC;QACzB,CAAC;QACD,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,OAAO,EAAE,UAAU,EAAE,IAAI,EAAE,WAAW,EAAE,WAAW,CAAC,IAAI,EAAE,CAAC,CAAC;IACrF,CAAC;IAED,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAChC,MAAM,KAAK,CAAC,GAAG,CAAC,oBAAoB,EAAE,aAAa,CAAC;YAClD,UAAU,EAAE,CAAC,IAAI,CAAC,GAAG,IAAI,IAAI,IAAI,EAAE,CAAC,CAAC,WAAW,EAAE;YAClD,QAAQ,EAAE,MAAM,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,EAAE,OAAO,EAAE,UAAU,EAAE,EAAE,EAAE,CAAC,CAAC,EAAE,OAAO,EAAE,UAAU,EAAE,CAAC,CAAC;YACrF,MAAM,EAAE,MAAM,CAAC,MAAM;SACtB,CAAC,CAAC,CAAC;IACN,CAAC;IACD,OAAO,MAAM,CAAC;AAChB,CAAC"}
|