pm-jev 2026.10.5 → 2026.10.28
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -3
- package/README.md +60 -2
- package/calibration/dedupe-report-v1.json +3 -3
- package/calibration/dedupe-report-v2.json +2 -2
- package/calibration/type-report-v1.json +12 -7
- package/dist/calibration-drift.d.ts +88 -0
- package/dist/calibration-drift.d.ts.map +1 -0
- package/dist/calibration-drift.js +135 -0
- package/dist/calibration-drift.js.map +1 -0
- package/dist/calibration.d.ts +7 -1
- package/dist/calibration.d.ts.map +1 -1
- package/dist/calibration.js +21 -15
- package/dist/calibration.js.map +1 -1
- package/dist/cost-policy.d.ts.map +1 -1
- package/dist/cost-policy.js +13 -10
- package/dist/cost-policy.js.map +1 -1
- package/dist/counterfactual-command.d.ts +1 -1
- package/dist/counterfactual.d.ts.map +1 -1
- package/dist/counterfactual.js +2 -1
- package/dist/counterfactual.js.map +1 -1
- package/dist/decision-calibration.d.ts.map +1 -1
- package/dist/decision-calibration.js +18 -11
- package/dist/decision-calibration.js.map +1 -1
- package/dist/dedupe-evaluation.d.ts +7 -3
- package/dist/dedupe-evaluation.d.ts.map +1 -1
- package/dist/dedupe-evaluation.js +34 -20
- package/dist/dedupe-evaluation.js.map +1 -1
- package/dist/dedupe.d.ts +3 -1
- package/dist/dedupe.d.ts.map +1 -1
- package/dist/dedupe.js +26 -15
- package/dist/dedupe.js.map +1 -1
- package/dist/drift-command.d.ts +14 -0
- package/dist/drift-command.d.ts.map +1 -0
- package/dist/drift-command.js +47 -0
- package/dist/drift-command.js.map +1 -0
- package/dist/duplicate-quality.d.ts +10 -2
- package/dist/duplicate-quality.d.ts.map +1 -1
- package/dist/duplicate-quality.js +34 -15
- package/dist/duplicate-quality.js.map +1 -1
- package/dist/evaluation-command.d.ts.map +1 -1
- package/dist/evaluation-command.js +4 -3
- package/dist/evaluation-command.js.map +1 -1
- package/dist/exact.d.ts +5 -0
- package/dist/exact.d.ts.map +1 -0
- package/dist/exact.js +3 -0
- package/dist/exact.js.map +1 -0
- package/dist/exemplar-retrieval.d.ts +0 -11
- package/dist/exemplar-retrieval.d.ts.map +1 -1
- package/dist/exemplar-retrieval.js +2 -1
- package/dist/exemplar-retrieval.js.map +1 -1
- package/dist/index.d.ts +22 -3
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +43 -20
- package/dist/index.js.map +1 -1
- package/dist/policy-replay.d.ts.map +1 -1
- package/dist/policy-replay.js +4 -3
- package/dist/policy-replay.js.map +1 -1
- package/dist/threshold.d.ts +56 -0
- package/dist/threshold.d.ts.map +1 -0
- package/dist/threshold.js +194 -0
- package/dist/threshold.js.map +1 -0
- package/dist/type-calibration.d.ts +4 -1
- package/dist/type-calibration.d.ts.map +1 -1
- package/dist/type-calibration.js +9 -6
- package/dist/type-calibration.js.map +1 -1
- package/docs/calibration-drift-evidence.json +95 -0
- package/docs/calibration-drift.md +139 -0
- package/docs/duplicate-quality.md +7 -0
- package/docs/exact-cutoffs.md +74 -0
- package/docs/threshold-domain.md +151 -0
- package/examples/calibration-drift.json +186 -0
- package/manifest.json +2 -2
- package/package.json +20 -9
package/CHANGELOG.md
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
-
## 2026.10.
|
|
3
|
+
## 2026.10.28 - 2026-10-28
|
|
4
4
|
|
|
5
5
|
### Added
|
|
6
6
|
|
|
7
|
-
-
|
|
7
|
+
- Detect calibration population and explicit-label drift without inference ([pm-jev-t1h1](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/features/pm-jev-t1h1.toon))
|
|
8
8
|
- Compare frozen decision proposals before adopting a local policy ([pm-jev-7esl](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/features/pm-jev-7esl.toon))
|
|
9
9
|
- Explain typed decisions by bounded evidence removal ([pm-jev-ygvl](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/features/pm-jev-ygvl.toon))
|
|
10
10
|
- Select measured cost-sensitive abstention policies ([pm-jev-do7y](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/features/pm-jev-do7y.toon))
|
|
@@ -20,6 +20,7 @@
|
|
|
20
20
|
- pm-jev ask command: caller-defined typed questions against an item ([pm-jev-2e0n](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/features/pm-jev-2e0n.toon))
|
|
21
21
|
- pm-jev gate command: calibrated noul exit gate for agents and CI ([pm-jev-nn3x](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/features/pm-jev-nn3x.toon))
|
|
22
22
|
- pm-jev models and doctor commands ([pm-jev-f8t5](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/features/pm-jev-f8t5.toon))
|
|
23
|
+
- Compact decision output for agent context budgets ([pm-jev-qriu](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/features/pm-jev-qriu.toon))
|
|
23
24
|
- Batch open backlog triage with bounded streaming and stable receipts ([pm-jev-k8qe](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/features/pm-jev-k8qe.toon))
|
|
24
25
|
|
|
25
26
|
### Changed
|
|
@@ -29,7 +30,6 @@
|
|
|
29
30
|
|
|
30
31
|
### Fixed
|
|
31
32
|
|
|
32
|
-
- Gate outcomes need explicit proposition and expected-result semantics ([pm-jev-43xa](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/issues/pm-jev-43xa.toon))
|
|
33
33
|
- Resolve development glob toolchain audit findings without downgrading fleet gates ([pm-jev-2rp1](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/issues/pm-jev-2rp1.toon))
|
|
34
34
|
- Preserve unknown token usage after TypeSafe SDK update ([pm-jev-ls5n](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/issues/pm-jev-ls5n.toon))
|
|
35
35
|
- Name evaluation source failures and support local tracker reads without GitHub ([pm-jev-0s07](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/issues/pm-jev-0s07.toon))
|
|
@@ -38,8 +38,11 @@
|
|
|
38
38
|
- Synthetic threshold calibration dataset and local harness ([pm-jev-6uoz](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/issues/pm-jev-6uoz.toon))
|
|
39
39
|
- Port best-effort CI approval and release-notes resume into the release workflow (Greptile PR 1) ([pm-jev-uqu6](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/issues/pm-jev-uqu6.toon))
|
|
40
40
|
- Address PR 1 eight CodeRabbit review findings ([pm-jev-3zuo](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/issues/pm-jev-3zuo.toon))
|
|
41
|
+
- Gate outcomes need explicit proposition and expected-result semantics ([pm-jev-43xa](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/issues/pm-jev-43xa.toon))
|
|
41
42
|
|
|
42
43
|
### Other
|
|
43
44
|
|
|
45
|
+
- Stop Hound from linting generated dist/ output ([pm-jev-hound-generated-dist](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/chores/pm-jev-hound-generated-dist.toon))
|
|
46
|
+
- Release readiness behind PM_RELEASE_APPROVED ([pm-jev-release-gate](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/chores/pm-jev-release-gate.toon))
|
|
44
47
|
- Preserve Unicode exemplar retrieval and UTF-8 prompt budgets ([pm-jev-sdk-unicode](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/tasks/pm-jev-sdk-unicode.toon))
|
|
45
48
|
- pm-jev: typed local-first System One decisions for pm items ([pm-jev-0w3q](https://github.com/unbraind/pm-jev/blob/main/.agents/pm/epics/pm-jev-0w3q.toon))
|
package/README.md
CHANGED
|
@@ -11,6 +11,32 @@ model, not a coding or text-generation model. Code handles candidate ranking, pr
|
|
|
11
11
|
mapping, probability thresholds, retries, deadlines and pm history. TypeSafe's RLCD
|
|
12
12
|
training is separate from the pm-rl evaluation work.
|
|
13
13
|
|
|
14
|
+
## Install and release approval
|
|
15
|
+
|
|
16
|
+
The first npm release is pending the maintainer's interactive bootstrap. After
|
|
17
|
+
that first publish, the maintainer configures npm trusted publishing for
|
|
18
|
+
`release.yml` and sets the repository variable `PM_RELEASE_APPROVED` to `true`.
|
|
19
|
+
The daily workflow and manual dispatch both require this variable and `main`.
|
|
20
|
+
Publishing uses OIDC with provenance; no stored npm token is needed.
|
|
21
|
+
|
|
22
|
+
Once published, install the package and compatible pm host, then activate the
|
|
23
|
+
extension in an initialized pm project:
|
|
24
|
+
|
|
25
|
+
```sh
|
|
26
|
+
npm install pm-jev @unbrained/pm-cli
|
|
27
|
+
npx --no-install pm package install ./node_modules/pm-jev
|
|
28
|
+
|
|
29
|
+
bun add pm-jev @unbrained/pm-cli
|
|
30
|
+
bunx --bun --no-install pm package install ./node_modules/pm-jev
|
|
31
|
+
|
|
32
|
+
pm package install npm:pm-jev
|
|
33
|
+
pm jev --help
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
The package exports its SDK through `main` and `types` and registers commands
|
|
37
|
+
with pm. It has no standalone `bin`: use `npx pm` or `bunx --bun pm` through
|
|
38
|
+
the installed host. npm/Bun dependency installation alone does not activate it.
|
|
39
|
+
|
|
14
40
|
## Install and check
|
|
15
41
|
|
|
16
42
|
Requirements: Node 22.18 or newer, pm-cli 2026.10.10 or newer, and Ollama with a
|
|
@@ -415,7 +441,7 @@ recommending the local flag that is already enabled; neither mode calls the mode
|
|
|
415
441
|
|
|
416
442
|
### Without git
|
|
417
443
|
|
|
418
|
-
Doctor, model discovery, triage, link, dedupe, ask, gate, explanations, cost-policy selection
|
|
444
|
+
Doctor, model discovery, triage, link, dedupe, ask, gate, explanations, cost-policy selection, offline policy replay and calibration drift checks work in plain trackers.
|
|
419
445
|
Evaluation and calibration work there with `--local-unverified` and explicit local
|
|
420
446
|
source directories; they require neither git nor GitHub. Default public-corpus
|
|
421
447
|
sources still require git, GitHub CLI authentication, verified public visibility
|
|
@@ -808,7 +834,7 @@ Generation and drift checks include closed and in-progress implementations;
|
|
|
808
834
|
tracker closure remains the orchestrator's verification step.
|
|
809
835
|
Scheduled and manual release runs are disabled in code by default. After approving the
|
|
810
836
|
first publish and configuring npm trusted publishing, the owner enables releases by
|
|
811
|
-
setting the repository variable `
|
|
837
|
+
setting the repository variable `PM_RELEASE_APPROVED` to `true`. The main-branch
|
|
812
838
|
gate remains required. Dependency installation, builds, checks and packing run with
|
|
813
839
|
`contents: read` and no persisted checkout credentials. The publishing job downloads
|
|
814
840
|
the verified tarball, runs no repository scripts or package dependency installation,
|
|
@@ -898,3 +924,35 @@ checkpoint directly without opening sources or contacting a model.
|
|
|
898
924
|
|
|
899
925
|
|
|
900
926
|
Validation: [public tracker receipt evidence](docs/partial-receipt-evidence.json).
|
|
927
|
+
|
|
928
|
+
Every workflow job granted write permissions, including CodeQL, Dependabot and
|
|
929
|
+
reusable workflow calls, requires the exact repository `unbraind/pm-jev`.
|
|
930
|
+
The approval variable controls GitHub releases; it does not impose a local npm
|
|
931
|
+
publishing hold. Maintainers run the prepublish checks and authenticate with npm
|
|
932
|
+
for local publication. The first local publish uses `--provenance=false`; daily
|
|
933
|
+
GitHub OIDC releases keep provenance enabled.
|
|
934
|
+
A maintainer enables the disabled workflow after merge with
|
|
935
|
+
`gh workflow enable release.yml --repo unbraind/pm-jev`. Workflow activation is
|
|
936
|
+
manual; the package does not install an automatic setup workflow.
|
|
937
|
+
|
|
938
|
+
## Calibration drift sentries
|
|
939
|
+
|
|
940
|
+
Compare operating populations and later explicit labels with a frozen calibration
|
|
941
|
+
cohort, without inference or tracker writes:
|
|
942
|
+
|
|
943
|
+
```sh
|
|
944
|
+
pm jev drift --cohorts examples/calibration-drift.json --json
|
|
945
|
+
```
|
|
946
|
+
|
|
947
|
+
The typed SDK exports `checkCalibrationDrift`, `parseDriftInput`,
|
|
948
|
+
`validateDriftReportNumbers`, `DriftRefusal` and their contracts. Reports bind the
|
|
949
|
+
exact model/question digests and predeclared thresholds, compare task/type/input
|
|
950
|
+
category distributions, and recommend abstention for drift or insufficient
|
|
951
|
+
labels. The compact TOON/JSON receipt omits observation details. This works
|
|
952
|
+
without Git. See the [strict schema, remedies and limitations](docs/calibration-drift.md),
|
|
953
|
+
[example](examples/calibration-drift.json), and
|
|
954
|
+
[read-only public-copy evidence](docs/calibration-drift-evidence.json).
|
|
955
|
+
|
|
956
|
+
Count-derived policy cutoffs use [shared exact rational arithmetic](docs/exact-cutoffs.md), including drift, empirical precision, F1 and cost selection. Drift threshold strings preserve the exact supplied decimal digits.
|
|
957
|
+
|
|
958
|
+
Fitted and applied cutoffs share [one threshold domain](docs/threshold-domain.md), with explicit disabled states, legacy Infinity compatibility and JSON/TOON reapplication laws. Fit lexical policies from raw pairs with `qualityObservation`; its exact portable ratio is also the runtime lexical guard input.
|
|
@@ -1347,7 +1347,7 @@
|
|
|
1347
1347
|
"fn": 1,
|
|
1348
1348
|
"precision": 0.30434782608695654,
|
|
1349
1349
|
"recall": 0.875,
|
|
1350
|
-
"f1": 0.
|
|
1350
|
+
"f1": 0.45161290322580644,
|
|
1351
1351
|
"precision_ci95": [
|
|
1352
1352
|
0.15604024453214022,
|
|
1353
1353
|
0.508657562687592
|
|
@@ -1375,7 +1375,7 @@
|
|
|
1375
1375
|
"fn": 7,
|
|
1376
1376
|
"precision": 0.2702702702702703,
|
|
1377
1377
|
"recall": 0.7407407407407407,
|
|
1378
|
-
"f1": 0.
|
|
1378
|
+
"f1": 0.39603960396039606,
|
|
1379
1379
|
"precision_ci95": [
|
|
1380
1380
|
0.1823022792207824,
|
|
1381
1381
|
0.3809124842856652
|
|
@@ -1406,7 +1406,7 @@
|
|
|
1406
1406
|
"fn": 13,
|
|
1407
1407
|
"precision": 0.3333333333333333,
|
|
1408
1408
|
"recall": 0.5185185185185185,
|
|
1409
|
-
"f1": 0.
|
|
1409
|
+
"f1": 0.4057971014492754,
|
|
1410
1410
|
"precision_ci95": [
|
|
1411
1411
|
0.21012467072357258,
|
|
1412
1412
|
0.484474926231662
|
|
@@ -2572,7 +2572,7 @@
|
|
|
2572
2572
|
"fn": 7,
|
|
2573
2573
|
"precision": 0.2702702702702703,
|
|
2574
2574
|
"recall": 0.7407407407407407,
|
|
2575
|
-
"f1": 0.
|
|
2575
|
+
"f1": 0.39603960396039606,
|
|
2576
2576
|
"precision_ci95": [
|
|
2577
2577
|
0.1823022792207824,
|
|
2578
2578
|
0.3809124842856652
|
|
@@ -2598,7 +2598,7 @@
|
|
|
2598
2598
|
"fn": 3,
|
|
2599
2599
|
"precision": 0.23809523809523808,
|
|
2600
2600
|
"recall": 0.625,
|
|
2601
|
-
"f1": 0.
|
|
2601
|
+
"f1": 0.3448275862068966,
|
|
2602
2602
|
"precision_ci95": [
|
|
2603
2603
|
0.10628013022636468,
|
|
2604
2604
|
0.4509117368295075
|
|
@@ -401,6 +401,11 @@
|
|
|
401
401
|
"targetPrecision": 0.95,
|
|
402
402
|
"minAccepted": 4,
|
|
403
403
|
"threshold": 0.05136322250317049,
|
|
404
|
+
"cutoff": {
|
|
405
|
+
"state": "enabled",
|
|
406
|
+
"numerator": "5136322250317049",
|
|
407
|
+
"denominator": "100000000000000000"
|
|
408
|
+
},
|
|
404
409
|
"curve": [
|
|
405
410
|
{
|
|
406
411
|
"threshold": 0,
|
|
@@ -417,7 +422,7 @@
|
|
|
417
422
|
"precision": 0.9583333333333334
|
|
418
423
|
},
|
|
419
424
|
{
|
|
420
|
-
"threshold": 0.
|
|
425
|
+
"threshold": 0.18923902111589694,
|
|
421
426
|
"accepted": 23,
|
|
422
427
|
"correct": 23,
|
|
423
428
|
"coverage": 0.9583333333333334,
|
|
@@ -431,7 +436,7 @@
|
|
|
431
436
|
"precision": 1
|
|
432
437
|
},
|
|
433
438
|
{
|
|
434
|
-
"threshold": 0.
|
|
439
|
+
"threshold": 0.2892897418205798,
|
|
435
440
|
"accepted": 21,
|
|
436
441
|
"correct": 21,
|
|
437
442
|
"coverage": 0.875,
|
|
@@ -487,7 +492,7 @@
|
|
|
487
492
|
"precision": 1
|
|
488
493
|
},
|
|
489
494
|
{
|
|
490
|
-
"threshold": 0.
|
|
495
|
+
"threshold": 0.7087908530213011,
|
|
491
496
|
"accepted": 13,
|
|
492
497
|
"correct": 13,
|
|
493
498
|
"coverage": 0.5416666666666666,
|
|
@@ -508,7 +513,7 @@
|
|
|
508
513
|
"precision": 1
|
|
509
514
|
},
|
|
510
515
|
{
|
|
511
|
-
"threshold": 0.
|
|
516
|
+
"threshold": 0.8148794285341366,
|
|
512
517
|
"accepted": 10,
|
|
513
518
|
"correct": 10,
|
|
514
519
|
"coverage": 0.4166666666666667,
|
|
@@ -522,21 +527,21 @@
|
|
|
522
527
|
"precision": 1
|
|
523
528
|
},
|
|
524
529
|
{
|
|
525
|
-
"threshold": 0.
|
|
530
|
+
"threshold": 0.8793367740289735,
|
|
526
531
|
"accepted": 8,
|
|
527
532
|
"correct": 8,
|
|
528
533
|
"coverage": 0.3333333333333333,
|
|
529
534
|
"precision": 1
|
|
530
535
|
},
|
|
531
536
|
{
|
|
532
|
-
"threshold": 0.
|
|
537
|
+
"threshold": 0.9072796440901532,
|
|
533
538
|
"accepted": 7,
|
|
534
539
|
"correct": 7,
|
|
535
540
|
"coverage": 0.2916666666666667,
|
|
536
541
|
"precision": 1
|
|
537
542
|
},
|
|
538
543
|
{
|
|
539
|
-
"threshold": 0.
|
|
544
|
+
"threshold": 0.9526519677632181,
|
|
540
545
|
"accepted": 6,
|
|
541
546
|
"correct": 6,
|
|
542
547
|
"coverage": 0.25,
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
/** Versioned population diagnostic; no provider or tracker mutation is involved. */
|
|
2
|
+
export declare const DRIFT_VERSION = "pm-jev-drift/1";
|
|
3
|
+
/** Exact frozen model weights and question catalog (or ordered catalog bundle) identity. */
|
|
4
|
+
export interface DriftIdentity {
|
|
5
|
+
model_digest: string;
|
|
6
|
+
question_digest: string;
|
|
7
|
+
}
|
|
8
|
+
/** Minimized evidence: input_bucket is a predeclared category, never raw input text. */
|
|
9
|
+
export interface DriftObservation {
|
|
10
|
+
id: string;
|
|
11
|
+
task: string;
|
|
12
|
+
type: string;
|
|
13
|
+
input_bucket: string;
|
|
14
|
+
predicted: string;
|
|
15
|
+
label: string | null;
|
|
16
|
+
}
|
|
17
|
+
/** A cohort and its exact decision identity; null labels mean no explicit later judgement. */
|
|
18
|
+
export interface DriftCohort {
|
|
19
|
+
identity: DriftIdentity;
|
|
20
|
+
observations: readonly DriftObservation[];
|
|
21
|
+
}
|
|
22
|
+
/** Freeze these inclusive thresholds before inspecting the current cohort. */
|
|
23
|
+
export interface DriftThresholds {
|
|
24
|
+
population_tv: number | string;
|
|
25
|
+
error_increase: number | string;
|
|
26
|
+
min_population: number;
|
|
27
|
+
min_labels: number;
|
|
28
|
+
}
|
|
29
|
+
/** Portable fixed evidence; public provenance is a caller assertion, not a publication permission. */
|
|
30
|
+
export interface DriftInput {
|
|
31
|
+
version: typeof DRIFT_VERSION;
|
|
32
|
+
provenance: "synthetic" | "public" | "local-unverified";
|
|
33
|
+
thresholds: DriftThresholds;
|
|
34
|
+
calibration: DriftCohort;
|
|
35
|
+
current: DriftCohort;
|
|
36
|
+
}
|
|
37
|
+
/** Machine-readable boundary refusals shared by SDK and command. */
|
|
38
|
+
export type DriftRefusalCode = "invalid_input" | "identity_mismatch" | "input_unreadable" | "invalid_output";
|
|
39
|
+
/** Content-free refusal with an actionable remedy, safe for compact agent output. */
|
|
40
|
+
export declare class DriftRefusal extends Error {
|
|
41
|
+
/** Stable routing code, independent of source text. */
|
|
42
|
+
readonly code: DriftRefusalCode;
|
|
43
|
+
/** Safe recovery instructions that do not reflect input values. */
|
|
44
|
+
readonly remedy: string;
|
|
45
|
+
/** Retain only typed diagnostics, never the rejected input. */
|
|
46
|
+
constructor(code: DriftRefusalCode, remedy: string);
|
|
47
|
+
}
|
|
48
|
+
/** Missing evidence is distinct from measured stability and an observed threshold crossing. */
|
|
49
|
+
export type DriftStatus = "stable" | "drift" | "insufficient_evidence";
|
|
50
|
+
/** Bounded diagnostic counts and rates; null means a required denominator was absent. */
|
|
51
|
+
export interface DriftReport {
|
|
52
|
+
ok: true;
|
|
53
|
+
version: typeof DRIFT_VERSION;
|
|
54
|
+
provenance: DriftInput["provenance"];
|
|
55
|
+
identity: DriftIdentity;
|
|
56
|
+
evidence_sha256: string;
|
|
57
|
+
calibration_sha256: string;
|
|
58
|
+
thresholds: DriftThresholds;
|
|
59
|
+
status: DriftStatus;
|
|
60
|
+
population: {
|
|
61
|
+
status: DriftStatus;
|
|
62
|
+
calibration_count: number;
|
|
63
|
+
current_count: number;
|
|
64
|
+
task_tv: number | null;
|
|
65
|
+
type_tv: number | null;
|
|
66
|
+
input_tv: number | null;
|
|
67
|
+
};
|
|
68
|
+
labels: {
|
|
69
|
+
status: DriftStatus;
|
|
70
|
+
calibration_count: number;
|
|
71
|
+
current_count: number;
|
|
72
|
+
calibration_coverage: number;
|
|
73
|
+
current_coverage: number;
|
|
74
|
+
calibration_error: number | null;
|
|
75
|
+
current_error: number | null;
|
|
76
|
+
error_increase: number | null;
|
|
77
|
+
};
|
|
78
|
+
recommendation: "retain_policy_and_monitor" | "abstain_and_recalibrate" | "abstain_pending_evidence";
|
|
79
|
+
inference_calls: 0;
|
|
80
|
+
mutation_authority: "none";
|
|
81
|
+
}
|
|
82
|
+
/** One strict boundary schema for both SDK inputs and command JSON; return a detached validated snapshot. */
|
|
83
|
+
export declare function parseDriftInput(raw: unknown): DriftInput;
|
|
84
|
+
/** Assert all emitted numbers have documented finite domains before exposing an ok result. */
|
|
85
|
+
export declare function validateDriftReportNumbers(report: DriftReport): DriftReport;
|
|
86
|
+
/** Compare frozen operating populations and later labels with zero inference, I/O or mutation. */
|
|
87
|
+
export declare function checkCalibrationDrift(raw: unknown): DriftReport;
|
|
88
|
+
//# sourceMappingURL=calibration-drift.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"calibration-drift.d.ts","sourceRoot":"","sources":["../calibration-drift.ts"],"names":[],"mappings":"AAIA,oFAAoF;AACpF,eAAO,MAAM,aAAa,mBAAmB,CAAC;AAC9C,4FAA4F;AAC5F,MAAM,WAAW,aAAa;IAAG,YAAY,EAAE,MAAM,CAAC;IAAC,eAAe,EAAE,MAAM,CAAA;CAAE;AAChF,wFAAwF;AACxF,MAAM,WAAW,gBAAgB;IAAG,EAAE,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAC;IAAC,YAAY,EAAE,MAAM,CAAC;IAAC,SAAS,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,GAAG,IAAI,CAAA;CAAE;AAC3I,8FAA8F;AAC9F,MAAM,WAAW,WAAW;IAAG,QAAQ,EAAE,aAAa,CAAC;IAAC,YAAY,EAAE,SAAS,gBAAgB,EAAE,CAAA;CAAE;AACnG,8EAA8E;AAC9E,MAAM,WAAW,eAAe;IAAG,aAAa,EAAE,MAAM,GAAG,MAAM,CAAC;IAAC,cAAc,EAAE,MAAM,GAAG,MAAM,CAAC;IAAC,cAAc,EAAE,MAAM,CAAC;IAAC,UAAU,EAAE,MAAM,CAAA;CAAE;AAChJ,sGAAsG;AACtG,MAAM,WAAW,UAAU;IAAG,OAAO,EAAE,OAAO,aAAa,CAAC;IAAC,UAAU,EAAE,WAAW,GAAG,QAAQ,GAAG,kBAAkB,CAAC;IAAC,UAAU,EAAE,eAAe,CAAC;IAAC,WAAW,EAAE,WAAW,CAAC;IAAC,OAAO,EAAE,WAAW,CAAA;CAAE;AACnM,oEAAoE;AACpE,MAAM,MAAM,gBAAgB,GAAG,eAAe,GAAG,mBAAmB,GAAG,kBAAkB,GAAG,gBAAgB,CAAC;AAC7G,qFAAqF;AACrF,qBAAa,YAAa,SAAQ,KAAK;IACrC,uDAAuD;IACvD,QAAQ,CAAC,IAAI,EAAE,gBAAgB,CAAC;IAChC,mEAAmE;IACnE,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,+DAA+D;IAC/D,YAAY,IAAI,EAAE,gBAAgB,EAAE,MAAM,EAAE,MAAM,EAKjD;CACF;AACD,+FAA+F;AAC/F,MAAM,MAAM,WAAW,GAAG,QAAQ,GAAG,OAAO,GAAG,uBAAuB,CAAC;AACvE,yFAAyF;AACzF,MAAM,WAAW,WAAW;IAC1B,EAAE,EAAE,IAAI,CAAC;IACT,OAAO,EAAE,OAAO,aAAa,CAAC;IAC9B,UAAU,EAAE,UAAU,CAAC,YAAY,CAAC,CAAC;IACrC,QAAQ,EAAE,aAAa,CAAC;IACxB,eAAe,EAAE,MAAM,CAAC;IACxB,kBAAkB,EAAE,MAAM,CAAC;IAC3B,UAAU,EAAE,eAAe,CAAC;IAC5B,MAAM,EAAE,WAAW,CAAC;IACpB,UAAU,EAAE;QAAE,MAAM,EAAE,WAAW,CAAC;QAAC,iBAAiB,EAAE,MAAM,CAAC;QAAC,aAAa,EAAE,MAAM,CAAC;QAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;QAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;QAAC,QAAQ,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,CAAC;IAC/J,MAAM,EAAE;QAAE,MAAM,EAAE,WAAW,CAAC;QAAC,iBAAiB,EAAE,MAAM,CAAC;QAAC,aAAa,EAAE,MAAM,CAAC;QAAC,oBAAoB,EAAE,MAAM,CAAC;QAAC,gBAAgB,EAAE,MAAM,CAAC;QAAC,iBAAiB,EAAE,MAAM,GAAG,IAAI,CAAC;QAAC,aAAa,EAAE,MAAM,GAAG,IAAI,CAAC;QAAC,cAAc,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,CAAC;IACzO,cAAc,EAAE,2BAA2B,GAAG,yBAAyB,GAAG,0BAA0B,CAAC;IACrG,eAAe,EAAE,CAAC,CAAC;IACnB,kBAAkB,EAAE,MAAM,CAAC;CAC5B;AAcD,6GAA6G;AAC7G,wBAAgB,eAAe,CAAC,GAAG,EAAE,OAAO,GAAG,UAAU,CA6BxD;AA0BD,8FAA8F;AAC9F,wBAAgB,0BAA0B,CAAC,MAAM,EAAE,WAAW,GAAG,WAAW,CAQ3E;AACD,kGAAkG;AAClG,wBAAgB,qBAAqB,CAAC,GAAG,EAAE,OAAO,GAAG,WAAW,CAgB/D"}
|
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
import { compareExact, decimal, displayExact, displayRatio, fraction, populationDistance, subtractExact } from "./exact.js";
|
|
2
|
+
import { evidenceHash } from "./decision-calibration.js";
|
|
3
|
+
/** Versioned population diagnostic; no provider or tracker mutation is involved. */
|
|
4
|
+
export const DRIFT_VERSION = "pm-jev-drift/1";
|
|
5
|
+
/** Content-free refusal with an actionable remedy, safe for compact agent output. */
|
|
6
|
+
export class DriftRefusal extends Error {
|
|
7
|
+
/** Stable routing code, independent of source text. */
|
|
8
|
+
code;
|
|
9
|
+
/** Safe recovery instructions that do not reflect input values. */
|
|
10
|
+
remedy;
|
|
11
|
+
/** Retain only typed diagnostics, never the rejected input. */
|
|
12
|
+
constructor(code, remedy) {
|
|
13
|
+
super(`Calibration drift check refused: ${code}`);
|
|
14
|
+
this.name = "DriftRefusal";
|
|
15
|
+
this.code = code;
|
|
16
|
+
this.remedy = remedy;
|
|
17
|
+
}
|
|
18
|
+
}
|
|
19
|
+
/** Require exact own fields so misspellings and accidental private payloads fail closed. */
|
|
20
|
+
function exact(value, keys) {
|
|
21
|
+
return value !== null && typeof value === "object" && !Array.isArray(value) && Object.keys(value).length === keys.length && keys.every(/** Inherited fields cannot satisfy the schema. */ /** Inherited fields cannot satisfy the schema. */ key => Object.hasOwn(value, key));
|
|
22
|
+
}
|
|
23
|
+
/** Bound categories without silently trimming or normalizing caller evidence. */
|
|
24
|
+
function category(value) { return typeof value === "string" && value.length > 0 && value.length <= 96 && value.trim() === value; }
|
|
25
|
+
/** Validate exact decimal thresholds without converting a supplied lexeme to binary. */
|
|
26
|
+
function validThreshold(value) {
|
|
27
|
+
if (!(typeof value === "number" && Number.isFinite(value) || typeof value === "string"))
|
|
28
|
+
return false;
|
|
29
|
+
try {
|
|
30
|
+
return compareExact(decimal(value), 0) > 0 && compareExact(decimal(value), 1) <= 0;
|
|
31
|
+
}
|
|
32
|
+
catch {
|
|
33
|
+
return false;
|
|
34
|
+
}
|
|
35
|
+
}
|
|
36
|
+
/** One strict boundary schema for both SDK inputs and command JSON; return a detached validated snapshot. */
|
|
37
|
+
export function parseDriftInput(raw) {
|
|
38
|
+
const data = raw;
|
|
39
|
+
let valid = exact(data, ["version", "provenance", "thresholds", "calibration", "current"]);
|
|
40
|
+
if (valid) {
|
|
41
|
+
const input = data;
|
|
42
|
+
const policy = input.thresholds;
|
|
43
|
+
valid = input.version === DRIFT_VERSION && ["synthetic", "public", "local-unverified"].includes(input.provenance) && exact(policy, ["population_tv", "error_increase", "min_population", "min_labels"]);
|
|
44
|
+
if (valid) {
|
|
45
|
+
valid = [policy.population_tv, policy.error_increase].every(/** Positive cutoffs avoid flagging unchanged cohorts at zero. */ /** Positive cutoffs avoid flagging unchanged cohorts at zero. */ value => validThreshold(value)) &&
|
|
46
|
+
[policy.min_population, policy.min_labels].every(/** Both populations require meaningful bounded denominators. */ /** Both populations require meaningful bounded denominators. */ value => Number.isSafeInteger(value) && value >= 2 && value <= 5000);
|
|
47
|
+
}
|
|
48
|
+
for (const cohort of [input.calibration, input.current]) {
|
|
49
|
+
if (!exact(cohort, ["identity", "observations"])) {
|
|
50
|
+
valid = false;
|
|
51
|
+
break;
|
|
52
|
+
}
|
|
53
|
+
const identity = cohort.identity;
|
|
54
|
+
if (!exact(identity, ["model_digest", "question_digest"]) || ![identity.model_digest, identity.question_digest].every(/** Bind SHA-256 identities, never mutable model tags. */ /** Bind SHA-256 identities, never mutable model tags. */ value => typeof value === "string" && /^[a-f0-9]{64}$/.test(value))) {
|
|
55
|
+
valid = false;
|
|
56
|
+
break;
|
|
57
|
+
}
|
|
58
|
+
const rows = cohort.observations;
|
|
59
|
+
if (!Array.isArray(rows) || rows.length > 5000 || Object.keys(rows).length !== rows.length) {
|
|
60
|
+
valid = false;
|
|
61
|
+
break;
|
|
62
|
+
}
|
|
63
|
+
const ids = new Set();
|
|
64
|
+
for (const [index, row] of rows.entries()) {
|
|
65
|
+
if (!Object.hasOwn(rows, index) || !exact(row, ["id", "task", "type", "input_bucket", "predicted", "label"]) || ![row.id, row.task, row.type, row.input_bucket, row.predicted].every(category) || !(row.label === null || category(row.label)) || ids.has(row.id)) {
|
|
66
|
+
valid = false;
|
|
67
|
+
break;
|
|
68
|
+
}
|
|
69
|
+
ids.add(row.id);
|
|
70
|
+
}
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
if (!valid)
|
|
74
|
+
throw new DriftRefusal("invalid_input", "Use pm-jev-drift/1 with exact documented fields, dense cohorts of at most 5000 unique rows, SHA-256 identities, bounded categories, finite thresholds in (0,1], and integer minima in [2,5000].");
|
|
75
|
+
const input = data;
|
|
76
|
+
return { version: DRIFT_VERSION, provenance: input.provenance,
|
|
77
|
+
thresholds: { population_tv: input.thresholds.population_tv, error_increase: input.thresholds.error_increase, min_population: input.thresholds.min_population, min_labels: input.thresholds.min_labels },
|
|
78
|
+
calibration: canonicalCohort(input.calibration), current: canonicalCohort(input.current) };
|
|
79
|
+
}
|
|
80
|
+
/** Canonical field and row order makes evidence hashes independent of JSON layout. */
|
|
81
|
+
function canonicalCohort(cohort) {
|
|
82
|
+
return { identity: { model_digest: cohort.identity.model_digest, question_digest: cohort.identity.question_digest },
|
|
83
|
+
observations: [...cohort.observations].sort(/** Stable ids define evidence enumeration. */ (a, b) => a.id < b.id ? -1 : 1)
|
|
84
|
+
.map(/** Copy only the validated minimized contract fields. */ /** Copy only the validated minimized contract fields. */ row => ({ id: row.id, task: row.task, type: row.type, input_bucket: row.input_bucket, predicted: row.predicted, label: row.label })) };
|
|
85
|
+
}
|
|
86
|
+
/** Total variation on a predeclared category union: zero is identical, one is disjoint. */
|
|
87
|
+
function totalVariation(a, b, key) {
|
|
88
|
+
if (a.length === 0 || b.length === 0)
|
|
89
|
+
return null;
|
|
90
|
+
const counts = new Map();
|
|
91
|
+
for (const [index, rows] of [a, b].entries()) {
|
|
92
|
+
for (const row of rows) {
|
|
93
|
+
const pair = counts.get(row[key]) ?? [0, 0];
|
|
94
|
+
pair[index]++;
|
|
95
|
+
counts.set(row[key], pair);
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
return populationDistance([...counts.values()].map(/** Keep calibration category counts exact. */ /** Keep calibration category counts exact. */ pair => pair[0]), [...counts.values()].map(/** Keep current category counts exact. */ /** Keep current category counts exact. */ pair => pair[1]));
|
|
99
|
+
}
|
|
100
|
+
/** Observe only explicit labels; predictions and status names never supply ground truth. */
|
|
101
|
+
function labelMetrics(rows) {
|
|
102
|
+
const labelled = rows.filter(/** A missing judgement cannot count as correct. */ /** A missing judgement cannot count as correct. */ row => row.label !== null);
|
|
103
|
+
return { count: labelled.length, coverage: rows.length === 0 ? 0 : displayRatio(labelled.length, rows.length),
|
|
104
|
+
error: labelled.length === 0 ? null : fraction(labelled.filter(/** Compare caller's explicit outcome with its recorded prediction. */ /** Compare caller's explicit outcome with its recorded prediction. */ row => row.label !== row.predicted).length, labelled.length) };
|
|
105
|
+
}
|
|
106
|
+
/** Assert all emitted numbers have documented finite domains before exposing an ok result. */
|
|
107
|
+
export function validateDriftReportNumbers(report) {
|
|
108
|
+
const { population: p, labels: l } = report;
|
|
109
|
+
const counts = [p.calibration_count, p.current_count, l.calibration_count, l.current_count];
|
|
110
|
+
const rates = [p.task_tv, p.type_tv, p.input_tv, l.calibration_coverage, l.current_coverage, l.calibration_error, l.current_error];
|
|
111
|
+
if (!counts.every(/** Counts cannot exceed the schema's cohort bound. */ /** Counts cannot exceed the schema's cohort bound. */ value => Number.isSafeInteger(value) && value >= 0 && value <= 5000) || !rates.every(/** Null explicitly denotes unavailable evidence. */ /** Null explicitly denotes unavailable evidence. */ value => value === null || Number.isFinite(value) && value >= 0 && value <= 1) || !(l.error_increase === null || Number.isFinite(l.error_increase) && l.error_increase >= -1 && l.error_increase <= 1)) {
|
|
112
|
+
throw new DriftRefusal("invalid_output", "Do not adopt this diagnostic; report a numeric invariant failure and retain the frozen evidence for inspection.");
|
|
113
|
+
}
|
|
114
|
+
return report;
|
|
115
|
+
}
|
|
116
|
+
/** Compare frozen operating populations and later labels with zero inference, I/O or mutation. */
|
|
117
|
+
export function checkCalibrationDrift(raw) {
|
|
118
|
+
const data = parseDriftInput(raw);
|
|
119
|
+
const { calibration: a, current: b, thresholds: t } = data;
|
|
120
|
+
if (a.identity.model_digest !== b.identity.model_digest || a.identity.question_digest !== b.identity.question_digest)
|
|
121
|
+
throw new DriftRefusal("identity_mismatch", "Collect a separate calibration cohort for the exact current model and question digest; do not transfer thresholds across identities.");
|
|
122
|
+
const task = totalVariation(a.observations, b.observations, "task");
|
|
123
|
+
const type = totalVariation(a.observations, b.observations, "type");
|
|
124
|
+
const input = totalVariation(a.observations, b.observations, "input_bucket");
|
|
125
|
+
const populationStatus = compareExact(Math.min(a.observations.length, b.observations.length), t.min_population) < 0 ? "insufficient_evidence" : [task, type, input].some(/** Thresholds are inclusive and frozen in the input. */ /** Thresholds are inclusive and frozen in the input. */ value => compareExact(value, t.population_tv) >= 0) ? "drift" : "stable";
|
|
126
|
+
const ca = labelMetrics(a.observations), cb = labelMetrics(b.observations);
|
|
127
|
+
const increase = ca.error === null || cb.error === null ? null : subtractExact(cb.error, ca.error);
|
|
128
|
+
const labelStatus = compareExact(Math.min(ca.count, cb.count), t.min_labels) < 0 ? "insufficient_evidence" : compareExact(increase, t.error_increase) >= 0 ? "drift" : "stable";
|
|
129
|
+
const status = populationStatus === "drift" || labelStatus === "drift" ? "drift" : populationStatus === "insufficient_evidence" || labelStatus === "insufficient_evidence" ? "insufficient_evidence" : "stable";
|
|
130
|
+
return validateDriftReportNumbers({ ok: true, version: DRIFT_VERSION, provenance: data.provenance, identity: a.identity, evidence_sha256: evidenceHash(data), calibration_sha256: evidenceHash(a), thresholds: t, status,
|
|
131
|
+
population: { status: populationStatus, calibration_count: a.observations.length, current_count: b.observations.length, task_tv: task === null ? null : displayExact(task), type_tv: type === null ? null : displayExact(type), input_tv: input === null ? null : displayExact(input) },
|
|
132
|
+
labels: { status: labelStatus, calibration_count: ca.count, current_count: cb.count, calibration_coverage: ca.coverage, current_coverage: cb.coverage, calibration_error: ca.error === null ? null : displayExact(ca.error), current_error: cb.error === null ? null : displayExact(cb.error), error_increase: increase === null ? null : displayExact(increase) },
|
|
133
|
+
recommendation: status === "drift" ? "abstain_and_recalibrate" : status === "insufficient_evidence" ? "abstain_pending_evidence" : "retain_policy_and_monitor", inference_calls: 0, mutation_authority: "none" });
|
|
134
|
+
}
|
|
135
|
+
//# sourceMappingURL=calibration-drift.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"calibration-drift.js","sourceRoot":"","sources":["../calibration-drift.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,OAAO,EAAE,YAAY,EAAE,YAAY,EAAE,QAAQ,EAAE,kBAAkB,EAAE,aAAa,EAAE,MAAM,YAAY,CAAC;AAE5H,OAAO,EAAE,YAAY,EAAE,MAAM,2BAA2B,CAAC;AAEzD,oFAAoF;AACpF,MAAM,CAAC,MAAM,aAAa,GAAG,gBAAgB,CAAC;AAa9C,qFAAqF;AACrF,MAAM,OAAO,YAAa,SAAQ,KAAK;IACrC,uDAAuD;IAC9C,IAAI,CAAmB;IAChC,mEAAmE;IAC1D,MAAM,CAAS;IACxB,+DAA+D;IAC/D,YAAY,IAAsB,EAAE,MAAc;QAChD,KAAK,CAAC,oCAAoC,IAAI,EAAE,CAAC,CAAC;QAClD,IAAI,CAAC,IAAI,GAAG,cAAc,CAAC;QAC3B,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;QACjB,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC;IACvB,CAAC;CACF;AAoBD,4FAA4F;AAC5F,SAAS,KAAK,CAAC,KAAc,EAAE,IAAuB;IACpD,OAAO,KAAK,KAAK,IAAI,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,MAAM,KAAK,IAAI,CAAC,MAAM,IAAI,IAAI,CAAC,KAAK,CAAC,kDAAkD,CAAC,AAAnD,kDAAkD,CAAC,GAAG,CAAC,EAAE,CAAC,MAAM,CAAC,MAAM,CAAC,KAAK,EAAE,GAAG,CAAC,CAAC,CAAC;AAC9N,CAAC;AACD,iFAAiF;AACjF,SAAS,QAAQ,CAAC,KAAc,IAAa,OAAO,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC,IAAI,KAAK,CAAC,MAAM,IAAI,EAAE,IAAI,KAAK,CAAC,IAAI,EAAE,KAAK,KAAK,CAAC,CAAC,CAAC;AACpJ,wFAAwF;AACxF,SAAS,cAAc,CAAC,KAAc;IACpC,IAAI,CAAC,CAAC,OAAO,KAAK,KAAK,QAAQ,IAAI,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,IAAI,OAAO,KAAK,KAAK,QAAQ,CAAC;QAAE,OAAO,KAAK,CAAC;IACtG,IAAI,CAAC;QAAC,OAAO,YAAY,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,GAAG,CAAC,IAAI,YAAY,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC;IAAC,CAAC;IAC3F,MAAM,CAAC;QAAC,OAAO,KAAK,CAAC;IAAC,CAAC;AACzB,CAAC;AACD,6GAA6G;AAC7G,MAAM,UAAU,eAAe,CAAC,GAAY;IAC1C,MAAM,IAAI,GAAG,GAAwB,CAAC;IACtC,IAAI,KAAK,GAAG,KAAK,CAAC,IAAI,EAAE,CAAC,SAAS,EAAE,YAAY,EAAE,YAAY,EAAE,aAAa,EAAE,SAAS,CAAC,CAAC,CAAC;IAC3F,IAAI,KAAK,EAAE,CAAC;QACV,MAAM,KAAK,GAAG,IAAkB,CAAC;QACjC,MAAM,MAAM,GAAG,KAAK,CAAC,UAAU,CAAC;QAChC,KAAK,GAAG,KAAK,CAAC,OAAO,KAAK,aAAa,IAAI,CAAC,WAAW,EAAE,QAAQ,EAAE,kBAAkB,CAAC,CAAC,QAAQ,CAAC,KAAK,CAAC,UAAU,CAAC,IAAI,KAAK,CAAC,MAAM,EAAE,CAAC,eAAe,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,YAAY,CAAC,CAAC,CAAC;QACxM,IAAI,KAAK,EAAE,CAAC;YACV,KAAK,GAAG,CAAC,MAAM,CAAC,aAAa,EAAE,MAAM,CAAC,cAAc,CAAC,CAAC,KAAK,CAAC,iEAAiE,CAAC,AAAlE,iEAAiE,CAAC,KAAK,CAAC,EAAE,CAAC,cAAc,CAAC,KAAK,CAAC,CAAC;gBAC3J,CAAC,MAAM,CAAC,cAAc,EAAE,MAAM,CAAC,UAAU,CAAC,CAAC,KAAK,CAAC,gEAAgE,CAAC,AAAjE,gEAAgE,CAAC,KAAK,CAAC,EAAE,CAAC,MAAM,CAAC,aAAa,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,IAAI,KAAK,IAAI,IAAI,CAAC,CAAC;QAC3L,CAAC;QACD,KAAK,MAAM,MAAM,IAAI,CAAC,KAAK,CAAC,WAAW,EAAE,KAAK,CAAC,OAAO,CAAC,EAAE,CAAC;YACxD,IAAI,CAAC,KAAK,CAAC,MAAM,EAAE,CAAC,UAAU,EAAE,cAAc,CAAC,CAAC,EAAE,CAAC;gBAAC,KAAK,GAAG,KAAK,CAAC;gBAAC,MAAM;YAAC,CAAC;YAC3E,MAAM,QAAQ,GAAG,MAAM,CAAC,QAAQ,CAAC;YACjC,IAAI,CAAC,KAAK,CAAC,QAAQ,EAAE,CAAC,cAAc,EAAE,iBAAiB,CAAC,CAAC,IAAI,CAAC,CAAC,QAAQ,CAAC,YAAY,EAAE,QAAQ,CAAC,eAAe,CAAC,CAAC,KAAK,CAAC,yDAAyD,CAAC,AAA1D,yDAAyD,CAAC,KAAK,CAAC,EAAE,CAAC,OAAO,KAAK,KAAK,QAAQ,IAAI,gBAAgB,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC;gBAAC,KAAK,GAAG,KAAK,CAAC;gBAAC,MAAM;YAAC,CAAC;YAC9Q,MAAM,IAAI,GAAG,MAAM,CAAC,YAAY,CAAC;YACjC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC,MAAM,GAAG,IAAI,IAAI,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,MAAM,KAAK,IAAI,CAAC,MAAM,EAAE,CAAC;gBAAC,KAAK,GAAG,KAAK,CAAC;gBAAC,MAAM;YAAC,CAAC;YACrH,MAAM,GAAG,GAAG,IAAI,GAAG,EAAU,CAAC;YAC9B,KAAK,MAAM,CAAC,KAAK,EAAE,GAAG,CAAC,IAAI,IAAI,CAAC,OAAO,EAAE,EAAE,CAAC;gBAC1C,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,cAAc,EAAE,WAAW,EAAE,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,EAAE,EAAE,GAAG,CAAC,IAAI,EAAE,GAAG,CAAC,IAAI,EAAE,GAAG,CAAC,YAAY,EAAE,GAAG,CAAC,SAAS,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,KAAK,KAAK,IAAI,IAAI,QAAQ,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC,IAAI,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,CAAC,EAAE,CAAC;oBAAC,KAAK,GAAG,KAAK,CAAC;oBAAC,MAAM;gBAAC,CAAC;gBAC5R,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;YAClB,CAAC;QACH,CAAC;IACH,CAAC;IACD,IAAI,CAAC,KAAK;QAAE,MAAM,IAAI,YAAY,CAAC,eAAe,EAAE,iMAAiM,CAAC,CAAC;IACvP,MAAM,KAAK,GAAG,IAAkB,CAAC;IACjC,OAAO,EAAE,OAAO,EAAE,aAAa,EAAE,UAAU,EAAE,KAAK,CAAC,UAAU;QAC3D,UAAU,EAAE,EAAE,aAAa,EAAE,KAAK,CAAC,UAAU,CAAC,aAAa,EAAE,cAAc,EAAE,KAAK,CAAC,UAAU,CAAC,cAAc,EAAE,cAAc,EAAE,KAAK,CAAC,UAAU,CAAC,cAAc,EAAE,UAAU,EAAE,KAAK,CAAC,UAAU,CAAC,UAAU,EAAE;QACxM,WAAW,EAAE,eAAe,CAAC,KAAK,CAAC,WAAW,CAAC,EAAE,OAAO,EAAE,eAAe,CAAC,KAAK,CAAC,OAAO,CAAC,EAAE,CAAC;AAC/F,CAAC;AACD,sFAAsF;AACtF,SAAS,eAAe,CAAC,MAAmB;IAC1C,OAAO,EAAE,QAAQ,EAAE,EAAE,YAAY,EAAE,MAAM,CAAC,QAAQ,CAAC,YAAY,EAAE,eAAe,EAAE,MAAM,CAAC,QAAQ,CAAC,eAAe,EAAE;QACjH,YAAY,EAAE,CAAC,GAAG,MAAM,CAAC,YAAY,CAAC,CAAC,IAAI,CAAC,8CAA8C,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;aACvH,GAAG,CAAC,yDAAyD,CAAC,AAA1D,yDAAyD,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,EAAE,EAAE,EAAE,GAAG,CAAC,EAAE,EAAE,IAAI,EAAE,GAAG,CAAC,IAAI,EAAE,IAAI,EAAE,GAAG,CAAC,IAAI,EAAE,YAAY,EAAE,GAAG,CAAC,YAAY,EAAE,SAAS,EAAE,GAAG,CAAC,SAAS,EAAE,KAAK,EAAE,GAAG,CAAC,KAAK,EAAE,CAAC,CAAC,EAAE,CAAC;AAC5M,CAAC;AACD,2FAA2F;AAC3F,SAAS,cAAc,CAAC,CAA8B,EAAE,CAA8B,EAAE,GAAqC;IAC3H,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC;IAClD,MAAM,MAAM,GAAG,IAAI,GAAG,EAA4B,CAAC;IACnD,KAAK,MAAM,CAAC,KAAK,EAAE,IAAI,CAAC,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC;QAC7C,KAAK,MAAM,GAAG,IAAI,IAAI,EAAE,CAAC;YACvB,MAAM,IAAI,GAAG,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;YAC5C,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;YACd,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,IAAI,CAAC,CAAC;QAC7B,CAAC;IACH,CAAC;IACD,OAAO,kBAAkB,CAAC,CAAC,GAAG,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC,GAAG,CAAC,8CAA8C,CAAC,AAA/C,8CAA8C,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,GAAG,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC,GAAG,CAAC,0CAA0C,CAAC,AAA3C,0CAA0C,CAAC,IAAI,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AAC5M,CAAC;AACD,4FAA4F;AAC5F,SAAS,YAAY,CAAC,IAAiC;IACrD,MAAM,QAAQ,GAAG,IAAI,CAAC,MAAM,CAAC,mDAAmD,CAAC,AAApD,mDAAmD,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,KAAK,KAAK,IAAI,CAAC,CAAC;IAC5G,OAAO,EAAE,KAAK,EAAE,QAAQ,CAAC,MAAM,EAAE,QAAQ,EAAE,IAAI,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,YAAY,CAAC,QAAQ,CAAC,MAAM,EAAE,IAAI,CAAC,MAAM,CAAC;QAC3G,KAAK,EAAE,QAAQ,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,QAAQ,CAAC,QAAQ,CAAC,MAAM,CAAC,sEAAsE,CAAC,AAAvE,sEAAsE,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,KAAK,KAAK,GAAG,CAAC,SAAS,CAAC,CAAC,MAAM,EAAE,QAAQ,CAAC,MAAM,CAAC,EAAE,CAAC;AACzM,CAAC;AACD,8FAA8F;AAC9F,MAAM,UAAU,0BAA0B,CAAC,MAAmB;IAC5D,MAAM,EAAE,UAAU,EAAE,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,GAAG,MAAM,CAAC;IAC5C,MAAM,MAAM,GAAG,CAAC,CAAC,CAAC,iBAAiB,EAAE,CAAC,CAAC,aAAa,EAAE,CAAC,CAAC,iBAAiB,EAAE,CAAC,CAAC,aAAa,CAAC,CAAC;IAC5F,MAAM,KAAK,GAAG,CAAC,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,QAAQ,EAAE,CAAC,CAAC,oBAAoB,EAAE,CAAC,CAAC,gBAAgB,EAAE,CAAC,CAAC,iBAAiB,EAAE,CAAC,CAAC,aAAa,CAAC,CAAC;IACnI,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,sDAAsD,CAAC,AAAvD,sDAAsD,CAAC,KAAK,CAAC,EAAE,CAAC,MAAM,CAAC,aAAa,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,IAAI,KAAK,IAAI,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,oDAAoD,CAAC,AAArD,oDAAoD,CAAC,KAAK,CAAC,EAAE,CAAC,KAAK,KAAK,IAAI,IAAI,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,IAAI,KAAK,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,cAAc,KAAK,IAAI,IAAI,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,cAAc,CAAC,IAAI,CAAC,CAAC,cAAc,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,cAAc,IAAI,CAAC,CAAC,EAAE,CAAC;QAC1Z,MAAM,IAAI,YAAY,CAAC,gBAAgB,EAAE,iHAAiH,CAAC,CAAC;IAC9J,CAAC;IACD,OAAO,MAAM,CAAC;AAChB,CAAC;AACD,kGAAkG;AAClG,MAAM,UAAU,qBAAqB,CAAC,GAAY;IAChD,MAAM,IAAI,GAAG,eAAe,CAAC,GAAG,CAAC,CAAC;IAClC,MAAM,EAAE,WAAW,EAAE,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE,UAAU,EAAE,CAAC,EAAE,GAAG,IAAI,CAAC;IAC3D,IAAI,CAAC,CAAC,QAAQ,CAAC,YAAY,KAAK,CAAC,CAAC,QAAQ,CAAC,YAAY,IAAI,CAAC,CAAC,QAAQ,CAAC,eAAe,KAAK,CAAC,CAAC,QAAQ,CAAC,eAAe;QAAE,MAAM,IAAI,YAAY,CAAC,mBAAmB,EAAE,sIAAsI,CAAC,CAAC;IAC1S,MAAM,IAAI,GAAG,cAAc,CAAC,CAAC,CAAC,YAAY,EAAE,CAAC,CAAC,YAAY,EAAE,MAAM,CAAC,CAAC;IACpE,MAAM,IAAI,GAAG,cAAc,CAAC,CAAC,CAAC,YAAY,EAAE,CAAC,CAAC,YAAY,EAAE,MAAM,CAAC,CAAC;IACpE,MAAM,KAAK,GAAG,cAAc,CAAC,CAAC,CAAC,YAAY,EAAE,CAAC,CAAC,YAAY,EAAE,cAAc,CAAC,CAAC;IAC7E,MAAM,gBAAgB,GAAgB,YAAY,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,YAAY,CAAC,MAAM,EAAE,CAAC,CAAC,YAAY,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,cAAc,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,uBAAuB,CAAC,CAAC,CAAC,CAAC,IAAI,EAAE,IAAI,EAAE,KAAK,CAAC,CAAC,IAAI,CAAC,wDAAwD,CAAC,AAAzD,wDAAwD,CAAC,KAAK,CAAC,EAAE,CAAC,YAAY,CAAC,KAAM,EAAE,CAAC,CAAC,aAAa,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,QAAQ,CAAC;IACzT,MAAM,EAAE,GAAG,YAAY,CAAC,CAAC,CAAC,YAAY,CAAC,EAAE,EAAE,GAAG,YAAY,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC;IAC3E,MAAM,QAAQ,GAAG,EAAE,CAAC,KAAK,KAAK,IAAI,IAAI,EAAE,CAAC,KAAK,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,aAAa,CAAC,EAAE,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC;IACnG,MAAM,WAAW,GAAgB,YAAY,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,uBAAuB,CAAC,CAAC,CAAC,YAAY,CAAC,QAAS,EAAE,CAAC,CAAC,cAAc,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,QAAQ,CAAC;IAC9L,MAAM,MAAM,GAAgB,gBAAgB,KAAK,OAAO,IAAI,WAAW,KAAK,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,gBAAgB,KAAK,uBAAuB,IAAI,WAAW,KAAK,uBAAuB,CAAC,CAAC,CAAC,uBAAuB,CAAC,CAAC,CAAC,QAAQ,CAAC;IAC7N,OAAO,0BAA0B,CAAC,EAAE,EAAE,EAAE,IAAI,EAAE,OAAO,EAAE,aAAa,EAAE,UAAU,EAAE,IAAI,CAAC,UAAU,EAAE,QAAQ,EAAE,CAAC,CAAC,QAAQ,EAAE,eAAe,EAAE,YAAY,CAAC,IAAI,CAAC,EAAE,kBAAkB,EAAE,YAAY,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,CAAC,EAAE,MAAM;QACtN,UAAU,EAAE,EAAE,MAAM,EAAE,gBAAgB,EAAE,iBAAiB,EAAE,CAAC,CAAC,YAAY,CAAC,MAAM,EAAE,aAAa,EAAE,CAAC,CAAC,YAAY,CAAC,MAAM,EAAE,OAAO,EAAE,IAAI,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,YAAY,CAAC,IAAI,CAAC,EAAE,OAAO,EAAE,IAAI,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,YAAY,CAAC,IAAI,CAAC,EAAE,QAAQ,EAAE,KAAK,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,YAAY,CAAC,KAAK,CAAC,EAAE;QACvR,MAAM,EAAE,EAAE,MAAM,EAAE,WAAW,EAAE,iBAAiB,EAAE,EAAE,CAAC,KAAK,EAAE,aAAa,EAAE,EAAE,CAAC,KAAK,EAAE,oBAAoB,EAAE,EAAE,CAAC,QAAQ,EAAE,gBAAgB,EAAE,EAAE,CAAC,QAAQ,EAAE,iBAAiB,EAAE,EAAE,CAAC,KAAK,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,YAAY,CAAC,EAAE,CAAC,KAAK,CAAC,EAAE,aAAa,EAAE,EAAE,CAAC,KAAK,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,YAAY,CAAC,EAAE,CAAC,KAAK,CAAC,EAAE,cAAc,EAAE,QAAQ,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,YAAY,CAAC,QAAQ,CAAC,EAAE;QAClW,cAAc,EAAE,MAAM,KAAK,OAAO,CAAC,CAAC,CAAC,yBAAyB,CAAC,CAAC,CAAC,MAAM,KAAK,uBAAuB,CAAC,CAAC,CAAC,0BAA0B,CAAC,CAAC,CAAC,2BAA2B,EAAE,eAAe,EAAE,CAAC,EAAE,kBAAkB,EAAE,MAAM,EAAE,CAAC,CAAC;AACtN,CAAC"}
|
package/dist/calibration.d.ts
CHANGED
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
import type { Threshold } from "./threshold.ts";
|
|
2
|
+
import type { ExactValue } from "./exact.ts";
|
|
1
3
|
import { TokenUsageTotals } from "./token-usage.ts";
|
|
2
4
|
import type { TokenUsage } from "./token-usage.ts";
|
|
3
5
|
import type { JevConfig } from "./index.ts";
|
|
@@ -49,12 +51,16 @@ export interface ThresholdDerivation {
|
|
|
49
51
|
readonly targetPrecision: number;
|
|
50
52
|
readonly minAccepted: number;
|
|
51
53
|
readonly threshold: number | null;
|
|
54
|
+
readonly cutoff: Threshold;
|
|
52
55
|
readonly curve: readonly ReliabilityPoint[];
|
|
53
56
|
}
|
|
54
57
|
/** Predict noul inclusively; confidence measures the probability of that label. */
|
|
55
58
|
export declare function observe(question: CalibrationQuestion, value: number, gateThreshold?: number): Observation;
|
|
56
59
|
/** Compute all attainable confidence cutoffs, retaining tied observations together. */
|
|
57
|
-
export declare function deriveThreshold(rows: readonly
|
|
60
|
+
export declare function deriveThreshold(rows: readonly {
|
|
61
|
+
readonly confidence: ExactValue;
|
|
62
|
+
readonly correct: boolean;
|
|
63
|
+
}[], targetPrecision?: number, minAccepted?: number): ThresholdDerivation;
|
|
58
64
|
/** Positive-class gate metrics differ from selective prediction correctness. */
|
|
59
65
|
export interface GatePoint {
|
|
60
66
|
readonly threshold: number;
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"calibration.d.ts","sourceRoot":"","sources":["../calibration.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"calibration.d.ts","sourceRoot":"","sources":["../calibration.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAChD,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAE7C,OAAO,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AACpD,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAInD,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,YAAY,CAAC;AAG5C,+EAA+E;AAC/E,eAAO,MAAM,eAAe,yBAAyB,CAAC;AACtD,sEAAsE;AACtE,eAAO,MAAM,QAAQ,YAAI,WAAW,EAAE,UAAU,EAAE,aAAa,EAAE,cAAc,CAAU,CAAC;AAC1F,4DAA4D;AAC5D,MAAM,MAAM,MAAM,GAAG,OAAO,QAAQ,CAAC,MAAM,CAAC,CAAC;AAC7C,+EAA+E;AAC/E,MAAM,WAAW,mBAAmB;IAClC,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,QAAQ,CAAC,KAAK,EAAE,aAAa,GAAG,YAAY,CAAC;IAC7C,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,QAAQ,EAAE,OAAO,CAAC;CAC5B;AACD,4EAA4E;AAC5E,MAAM,WAAW,kBAAkB;IACjC,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,SAAS,EAAE,SAAS,mBAAmB,EAAE,CAAC;CACpD;AACD,qFAAqF;AACrF,wBAAgB,eAAe,CAAC,IAAI,SAAW,GAAG,kBAAkB,CAoCnE;AAED,2EAA2E;AAC3E,wBAAgB,YAAY,CAAC,OAAO,EAAE,kBAAkB,GAAG,MAAM,CAEhE;AAED,oFAAoF;AACpF,wBAAgB,WAAW,CAAC,IAAI,EAAE,MAAM,GAAG,kBAAkB,CAO5D;AAED,wFAAwF;AACxF,MAAM,WAAW,WAAY,SAAQ,mBAAmB;IACtD,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,SAAS,EAAE,OAAO,CAAC;IAC5B,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;IAC5B,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAC;IAC1B,QAAQ,CAAC,KAAK,CAAC,EAAE,UAAU,CAAC;CAC7B;AACD,8EAA8E;AAC9E,MAAM,WAAW,gBAAgB;IAC/B,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;CACnC;AACD,oFAAoF;AACpF,MAAM,WAAW,mBAAmB;IAClC,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAC;IACjC,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IAClC,QAAQ,CAAC,MAAM,EAAE,SAAS,CAAC;IAC3B,QAAQ,CAAC,KAAK,EAAE,SAAS,gBAAgB,EAAE,CAAC;CAC7C;AAUD,mFAAmF;AACnF,wBAAgB,OAAO,CAAC,QAAQ,EAAE,mBAAmB,EAAE,KAAK,EAAE,MAAM,EAAE,aAAa,SAAM,GAAG,WAAW,CAKtG;AAED,uFAAuF;AACvF,wBAAgB,eAAe,CAAC,IAAI,EAAE,SAAS;IAAE,QAAQ,CAAC,UAAU,EAAE,UAAU,CAAC;IAAC,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAA;CAAE,EAAE,EAAE,eAAe,SAAO,EAAE,WAAW,SAAI,GAAG,mBAAmB,CAY7K;AAED,gFAAgF;AAChF,MAAM,WAAW,SAAS;IACxB,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,aAAa,EAAE,MAAM,CAAC;IAC/B,QAAQ,CAAC,aAAa,EAAE,MAAM,CAAC;IAC/B,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IAClC,QAAQ,CAAC,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;IAC/B,QAAQ,CAAC,QAAQ,EAAE,MAAM,GAAG,IAAI,CAAC;CAClC;AAED,kFAAkF;AAClF,wBAAgB,SAAS,CAAC,IAAI,EAAE,SAAS,WAAW,EAAE,EAAE,SAAS,EAAE,MAAM,GAAG,SAAS,CAepF;AAED,oFAAoF;AACpF,wBAAgB,aAAa,CAAC,IAAI,EAAE,SAAS,WAAW,EAAE,EAAE,eAAe,EAAE,MAAM,EAAE,WAAW,EAAE,MAAM;;;;;QAUpG,KAAK;QACL,QAAQ;QACR,QAAQ;QACR,SAAS;QACT,IAAI;QACJ,YAAY;;EAEf;AAED,4EAA4E;AAC5E,wBAAgB,WAAW,CAAC,OAAO,SAA2B,EAAE,KAAK,SAAY,GAAG,SAAS,CAK5F;AAED,oFAAoF;AACpF,wBAAsB,UAAU,CAAC,MAAM,EAAE,SAAS,GAAG,OAAO,CAAC,MAAM,CAAC,CAWnE;AAED,mFAAmF;AACnF,wBAAsB,SAAS,CAAC,OAAO,EAAE,kBAAkB,EAAE,OAAO,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,eAAe,SAAO,EAAE,WAAW,SAAI;;;;;;;;;;;;;;;;;;;;gBAhChI,KAAK;gBACL,QAAQ;gBACR,QAAQ;gBACR,SAAS;gBACT,IAAI;gBACJ,YAAY;;;;;;;;;;;;;;;;;;;GA2Df"}
|