eval-quality 0.2.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -12
- package/corpus/dev/README.md +3 -2
- package/corpus/dev/compile-seal-example/brief.json +1 -1
- package/corpus/dev/compile-seal-example/contract.json +1 -1
- package/corpus/dev/contracts/absent-collection-locations.json +1 -1
- package/corpus/dev/contracts/absent-sibling-groups.json +1 -1
- package/corpus/dev/contracts/absent-success-indicator.json +1 -1
- package/corpus/dev/contracts/empty-channel-roles.json +1 -1
- package/corpus/dev/contracts/empty-collection-locations.json +1 -1
- package/corpus/dev/contracts/empty-request-shapes.json +1 -1
- package/corpus/dev/contracts/empty-sibling-groups.json +1 -1
- package/corpus/dev/contracts/fragment-selection.json +1 -0
- package/corpus/dev/contracts/no-collection-quantifier.json +1 -1
- package/corpus/dev/contracts/no-operation-inventory.json +1 -1
- package/corpus/dev/contracts/no-read-back-relation.json +1 -1
- package/corpus/dev/contracts/no-state-change-marker.json +1 -1
- package/corpus/dev/contracts/no-type-violating-step.json +1 -1
- package/corpus/dev/contracts/per-key-split-oracles.json +1 -1
- package/corpus/dev/contracts/review-corpus.json +1 -0
- package/corpus/dev/contracts/satisfied-declarations.json +1 -1
- package/corpus/dev/contracts/single-required-response-key.json +1 -1
- package/corpus/dev/contracts/split-indicator-oracle.json +1 -1
- package/corpus/dev/contracts/unaddressed-parameter-sibling.json +1 -1
- package/corpus/dev/contracts/unnamed-reference-set.json +1 -1
- package/corpus/dev/contracts/wrong-cardinality-form.json +1 -1
- package/corpus/dev/index.json +1 -1
- package/dist/adapters/command-line-adapter.d.ts +44 -0
- package/dist/adapters/command-line-adapter.js +284 -0
- package/dist/adapters/command-target-policy.d.ts +36 -0
- package/dist/adapters/command-target-policy.js +33 -0
- package/dist/adapters/index.d.ts +2 -0
- package/dist/adapters/index.js +1 -0
- package/dist/cli/render.d.ts +2 -5
- package/dist/cli/render.js +51 -1
- package/dist/core/compile/bindings.d.ts +26 -7
- package/dist/core/compile/bindings.js +49 -32
- package/dist/core/compile/compile.d.ts +12 -0
- package/dist/core/compile/compile.js +28 -1
- package/dist/core/compile/excluded-content.d.ts +11 -0
- package/dist/core/compile/excluded-content.js +42 -0
- package/dist/core/compile/expression-legality.d.ts +2 -2
- package/dist/core/compile/expression-legality.js +58 -12
- package/dist/core/compile/interface-inventory.d.ts +34 -1
- package/dist/core/compile/interface-inventory.js +88 -11
- package/dist/core/compile/reachability.d.ts +12 -2
- package/dist/core/compile/reachability.js +123 -36
- package/dist/core/compile/schema-version.d.ts +2 -0
- package/dist/core/compile/schema-version.js +25 -0
- package/dist/core/compile/sensitivity-witness.d.ts +31 -12
- package/dist/core/compile/sensitivity-witness.js +110 -23
- package/dist/core/compile/step-reference.d.ts +2 -0
- package/dist/core/compile/step-reference.js +49 -0
- package/dist/core/coverage/operations.d.ts +62 -0
- package/dist/core/coverage/operations.js +57 -0
- package/dist/core/coverage/relevance.d.ts +4 -2
- package/dist/core/coverage/relevance.js +22 -23
- package/dist/core/coverage/satisfaction.d.ts +2 -2
- package/dist/core/coverage/satisfaction.js +73 -48
- package/dist/core/declared-inputs.d.ts +83 -4
- package/dist/core/declared-inputs.js +105 -8
- package/dist/core/evaluate/evidence-resolution.d.ts +9 -12
- package/dist/core/evaluate/evidence-resolution.js +70 -10
- package/dist/core/evaluate/operators.d.ts +22 -0
- package/dist/core/evaluate/operators.js +17 -1
- package/dist/core/evaluate/resolution.d.ts +14 -3
- package/dist/core/evaluate/resolution.js +63 -3
- package/dist/core/excluded-content.d.ts +65 -0
- package/dist/core/excluded-content.js +113 -0
- package/dist/core/failure-codes.d.ts +2 -2
- package/dist/core/failure-codes.js +5 -2
- package/dist/core/ingest/conditions.d.ts +1 -1
- package/dist/core/ingest/ingest.js +8 -0
- package/dist/core/preflight/plan.d.ts +7 -5
- package/dist/core/preflight/plan.js +75 -32
- package/dist/core/preflight/projection.d.ts +6 -3
- package/dist/core/preflight/projection.js +22 -2
- package/dist/core/preflight/reduce.js +47 -7
- package/dist/core/preflight/witness-evidence.d.ts +5 -5
- package/dist/core/preflight/witness-evidence.js +63 -20
- package/dist/core/schemas/artifact.d.ts +683 -35
- package/dist/core/schemas/constraint-ledger.js +11 -0
- package/dist/core/schemas/defect-signature.d.ts +438 -16
- package/dist/core/schemas/defect-signature.js +58 -10
- package/dist/core/schemas/eval-contract.d.ts +517 -12
- package/dist/core/schemas/eval-contract.js +18 -2
- package/dist/core/schemas/evidence-artifact.d.ts +14 -4
- package/dist/core/schemas/evidence-artifact.js +1 -1
- package/dist/core/schemas/interface.d.ts +741 -13
- package/dist/core/schemas/interface.js +120 -7
- package/dist/core/schemas/isolation-manifest.js +16 -9
- package/dist/core/schemas/plan.d.ts +217 -3
- package/dist/core/schemas/plan.js +22 -1
- package/dist/core/schemas/pointer.d.ts +47 -1
- package/dist/core/schemas/pointer.js +89 -8
- package/dist/core/schemas/port-messages.d.ts +194 -10
- package/dist/core/schemas/port-messages.js +73 -6
- package/dist/core/schemas/probe-body.d.ts +18 -0
- package/dist/core/schemas/probe-body.js +13 -0
- package/dist/core/schemas/probe-policy.d.ts +36 -0
- package/dist/core/schemas/probe-policy.js +44 -0
- package/dist/core/schemas/probe.d.ts +122 -3
- package/dist/core/schemas/probe.js +1 -1
- package/dist/core/schemas/sealed-run-record.d.ts +131 -9
- package/dist/core/schemas/sealed-run-record.js +56 -11
- package/dist/core/schemas/sensitivity-witness.d.ts +108 -6
- package/dist/core/schemas/sensitivity-witness.js +61 -5
- package/dist/core/score/bindings.d.ts +2 -2
- package/dist/core/score/bindings.js +25 -13
- package/dist/core/score/qualification.d.ts +5 -5
- package/dist/core/score/qualification.js +67 -37
- package/dist/core/score/quotation.d.ts +1 -24
- package/dist/core/score/quotation.js +29 -4
- package/dist/core/score/score.js +28 -2
- package/dist/core/score/strength.d.ts +9 -0
- package/dist/core/score/strength.js +35 -1
- package/dist/core/score/witness.d.ts +21 -0
- package/dist/core/score/witness.js +9 -4
- package/dist/core/seal/derived-reference.d.ts +2 -9
- package/dist/core/seal/derived-reference.js +62 -23
- package/dist/core/seal/plan-index.d.ts +18 -3
- package/dist/core/seal/plan-index.js +44 -8
- package/dist/index.d.ts +1 -1
- package/dist/index.js +1 -1
- package/dist/ports/environment-probe-port.d.ts +60 -4
- package/dist/testing/conformance.d.ts +2 -1
- package/dist/testing/conformance.js +1 -0
- package/dist/testing/index.d.ts +3 -3
- package/dist/testing/index.js +1 -1
- package/dist/testing/probe-conformance.d.ts +56 -5
- package/dist/testing/probe-conformance.js +196 -5
- package/package.json +1 -1
- package/schemas/eval-contract.schema.json +1356 -464
- package/schemas/evidence-artifact.schema.json +59 -29
- package/schemas/isolation-manifest.schema.json +17 -10
- package/schemas/probe.schema.json +197 -71
- package/schemas/rubric.schema.json +1 -1
- package/schemas/sealed-run-record.schema.json +296 -39
|
@@ -431,10 +431,90 @@ export declare const Probe: z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
|
431
431
|
relation: z.ZodType<import("./expression.ts").Expression, unknown, z.core.$ZodTypeInternals<import("./expression.ts").Expression, unknown>>;
|
|
432
432
|
}, z.core.$strict>>;
|
|
433
433
|
}, z.core.$strict>>;
|
|
434
|
-
defectSignature: z.ZodNullable<z.ZodObject<{
|
|
434
|
+
defectSignature: z.ZodNullable<z.ZodUnion<readonly [z.ZodObject<{
|
|
435
|
+
observableChannel: z.ZodEnum<{
|
|
436
|
+
artifact: "artifact";
|
|
437
|
+
"call-inputs": "call-inputs";
|
|
438
|
+
"exit-code": "exit-code";
|
|
439
|
+
"response-body": "response-body";
|
|
440
|
+
"response-headers": "response-headers";
|
|
441
|
+
"response-status": "response-status";
|
|
442
|
+
stderr: "stderr";
|
|
443
|
+
stdout: "stdout";
|
|
444
|
+
}>;
|
|
445
|
+
condition: z.ZodObject<{
|
|
446
|
+
selector: z.ZodObject<{
|
|
447
|
+
inputBinding: z.ZodObject<{
|
|
448
|
+
path: z.ZodNullable<z.ZodRecord<z.ZodString, z.ZodUnion<readonly [z.ZodObject<{
|
|
449
|
+
literal: z.ZodType<import("./primitives.ts").JsonValue, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonValue, unknown>>;
|
|
450
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
451
|
+
matcher: z.ZodEnum<{
|
|
452
|
+
any: "any";
|
|
453
|
+
"type-violating": "type-violating";
|
|
454
|
+
}>;
|
|
455
|
+
}, z.core.$strict>]>>>;
|
|
456
|
+
query: z.ZodNullable<z.ZodRecord<z.ZodString, z.ZodUnion<readonly [z.ZodObject<{
|
|
457
|
+
literal: z.ZodType<import("./primitives.ts").JsonValue, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonValue, unknown>>;
|
|
458
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
459
|
+
matcher: z.ZodEnum<{
|
|
460
|
+
any: "any";
|
|
461
|
+
"type-violating": "type-violating";
|
|
462
|
+
}>;
|
|
463
|
+
}, z.core.$strict>]>>>;
|
|
464
|
+
header: z.ZodNullable<z.ZodRecord<z.ZodString, z.ZodUnion<readonly [z.ZodObject<{
|
|
465
|
+
literal: z.ZodType<import("./primitives.ts").JsonValue, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonValue, unknown>>;
|
|
466
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
467
|
+
matcher: z.ZodEnum<{
|
|
468
|
+
any: "any";
|
|
469
|
+
"type-violating": "type-violating";
|
|
470
|
+
}>;
|
|
471
|
+
}, z.core.$strict>]>>>;
|
|
472
|
+
body: z.ZodNullable<z.ZodRecord<z.ZodString, z.ZodUnion<readonly [z.ZodObject<{
|
|
473
|
+
literal: z.ZodType<import("./primitives.ts").JsonValue, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonValue, unknown>>;
|
|
474
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
475
|
+
matcher: z.ZodEnum<{
|
|
476
|
+
any: "any";
|
|
477
|
+
"type-violating": "type-violating";
|
|
478
|
+
}>;
|
|
479
|
+
}, z.core.$strict>]>>>;
|
|
480
|
+
argument: z.ZodNullable<z.ZodRecord<z.ZodString, z.ZodUnion<readonly [z.ZodObject<{
|
|
481
|
+
literal: z.ZodType<import("./primitives.ts").JsonValue, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonValue, unknown>>;
|
|
482
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
483
|
+
matcher: z.ZodEnum<{
|
|
484
|
+
any: "any";
|
|
485
|
+
"type-violating": "type-violating";
|
|
486
|
+
}>;
|
|
487
|
+
}, z.core.$strict>]>>>;
|
|
488
|
+
option: z.ZodNullable<z.ZodRecord<z.ZodString, z.ZodUnion<readonly [z.ZodObject<{
|
|
489
|
+
literal: z.ZodType<import("./primitives.ts").JsonValue, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonValue, unknown>>;
|
|
490
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
491
|
+
matcher: z.ZodEnum<{
|
|
492
|
+
any: "any";
|
|
493
|
+
"type-violating": "type-violating";
|
|
494
|
+
}>;
|
|
495
|
+
}, z.core.$strict>]>>>;
|
|
496
|
+
environment: z.ZodNullable<z.ZodRecord<z.ZodString, z.ZodUnion<readonly [z.ZodObject<{
|
|
497
|
+
literal: z.ZodType<import("./primitives.ts").JsonValue, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonValue, unknown>>;
|
|
498
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
499
|
+
matcher: z.ZodEnum<{
|
|
500
|
+
any: "any";
|
|
501
|
+
"type-violating": "type-violating";
|
|
502
|
+
}>;
|
|
503
|
+
}, z.core.$strict>]>>>;
|
|
504
|
+
stdin: z.ZodNullable<z.ZodRecord<z.ZodString, z.ZodUnion<readonly [z.ZodObject<{
|
|
505
|
+
literal: z.ZodType<import("./primitives.ts").JsonValue, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonValue, unknown>>;
|
|
506
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
507
|
+
matcher: z.ZodEnum<{
|
|
508
|
+
any: "any";
|
|
509
|
+
"type-violating": "type-violating";
|
|
510
|
+
}>;
|
|
511
|
+
}, z.core.$strict>]>>>;
|
|
512
|
+
}, z.core.$strict>;
|
|
513
|
+
}, z.core.$strict>;
|
|
514
|
+
predicate: z.ZodType<import("./expression.ts").Expression, unknown, z.core.$ZodTypeInternals<import("./expression.ts").Expression, unknown>>;
|
|
515
|
+
}, z.core.$strict>;
|
|
435
516
|
interfaceKind: z.ZodEnum<{
|
|
436
517
|
api: "api";
|
|
437
|
-
cli: "cli";
|
|
438
518
|
mcp: "mcp";
|
|
439
519
|
web: "web";
|
|
440
520
|
}>;
|
|
@@ -448,7 +528,9 @@ export declare const Probe: z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
|
448
528
|
PUT: "PUT";
|
|
449
529
|
}>;
|
|
450
530
|
pathTemplate: z.ZodString;
|
|
531
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
451
532
|
observableChannel: z.ZodEnum<{
|
|
533
|
+
artifact: "artifact";
|
|
452
534
|
"call-inputs": "call-inputs";
|
|
453
535
|
"exit-code": "exit-code";
|
|
454
536
|
"response-body": "response-body";
|
|
@@ -492,10 +574,47 @@ export declare const Probe: z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
|
492
574
|
"type-violating": "type-violating";
|
|
493
575
|
}>;
|
|
494
576
|
}, z.core.$strict>]>>>;
|
|
577
|
+
argument: z.ZodNullable<z.ZodRecord<z.ZodString, z.ZodUnion<readonly [z.ZodObject<{
|
|
578
|
+
literal: z.ZodType<import("./primitives.ts").JsonValue, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonValue, unknown>>;
|
|
579
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
580
|
+
matcher: z.ZodEnum<{
|
|
581
|
+
any: "any";
|
|
582
|
+
"type-violating": "type-violating";
|
|
583
|
+
}>;
|
|
584
|
+
}, z.core.$strict>]>>>;
|
|
585
|
+
option: z.ZodNullable<z.ZodRecord<z.ZodString, z.ZodUnion<readonly [z.ZodObject<{
|
|
586
|
+
literal: z.ZodType<import("./primitives.ts").JsonValue, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonValue, unknown>>;
|
|
587
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
588
|
+
matcher: z.ZodEnum<{
|
|
589
|
+
any: "any";
|
|
590
|
+
"type-violating": "type-violating";
|
|
591
|
+
}>;
|
|
592
|
+
}, z.core.$strict>]>>>;
|
|
593
|
+
environment: z.ZodNullable<z.ZodRecord<z.ZodString, z.ZodUnion<readonly [z.ZodObject<{
|
|
594
|
+
literal: z.ZodType<import("./primitives.ts").JsonValue, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonValue, unknown>>;
|
|
595
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
596
|
+
matcher: z.ZodEnum<{
|
|
597
|
+
any: "any";
|
|
598
|
+
"type-violating": "type-violating";
|
|
599
|
+
}>;
|
|
600
|
+
}, z.core.$strict>]>>>;
|
|
601
|
+
stdin: z.ZodNullable<z.ZodRecord<z.ZodString, z.ZodUnion<readonly [z.ZodObject<{
|
|
602
|
+
literal: z.ZodType<import("./primitives.ts").JsonValue, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonValue, unknown>>;
|
|
603
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
604
|
+
matcher: z.ZodEnum<{
|
|
605
|
+
any: "any";
|
|
606
|
+
"type-violating": "type-violating";
|
|
607
|
+
}>;
|
|
608
|
+
}, z.core.$strict>]>>>;
|
|
495
609
|
}, z.core.$strict>;
|
|
496
610
|
}, z.core.$strict>;
|
|
497
611
|
predicate: z.ZodType<import("./expression.ts").Expression, unknown, z.core.$ZodTypeInternals<import("./expression.ts").Expression, unknown>>;
|
|
498
612
|
}, z.core.$strict>;
|
|
499
|
-
|
|
613
|
+
interfaceKind: z.ZodLiteral<"cli">;
|
|
614
|
+
invocation: z.ZodObject<{
|
|
615
|
+
executable: z.ZodString;
|
|
616
|
+
subcommandPath: z.ZodArray<z.ZodString>;
|
|
617
|
+
}, z.core.$strict>;
|
|
618
|
+
}, z.core.$strict>]>>;
|
|
500
619
|
}, z.core.$strict>], "expectedClean">;
|
|
501
620
|
export type Probe = z.infer<typeof Probe>;
|
|
@@ -82,5 +82,5 @@ export const Probe = z
|
|
|
82
82
|
])
|
|
83
83
|
.meta({
|
|
84
84
|
id: 'Probe',
|
|
85
|
-
description: "One corpus probe. Succeeds the prior-art `h0-ground-truth` schema per AD-24, carrying its system identifier, implementation digest, `expectedClean` flag, seeded defects, and rationale, and adding AD-9's probe class and AD-9's per-probe artifact and commit digests. Divergences: `implementationSha` becomes `implementationDigest`, `taskId` does not survive because the probe pins what it describes by digest, and `expectedGate` does not survive because AD-40 makes detection a signature match rather than a verdict comparison and AD-7 keeps comparisons inside the dominance vector, so carrying an expected gate would invite a comparison the architecture forbids. Two constructions landed here together under one BREAKING `schemaVersion` bump: AD-9's per-class QUALIFICATION record, as a five-route tagged union required on every branch, and AD-40's machine-readable DEFECT SIGNATURE on the `expectedClean: false` branch, carrying the interface kind, the home operation as a method and a path template, the observable channel, and the discriminating condition. Both are required rather than optional, which is what makes the bump breaking under AD-11: every corpus written against version 1 fails to parse. What the schema still does not decide is stated rather than hidden. AD-9's \"an unqualified probe cannot enter a sealed set\" is a corpus-construction invariant enforced by the qualification gate in `core/score/qualification.ts`, not by this schema: all eight class-and-`expectedClean` pairings parse, a route incompatible with the pair parses, and a signature-less non-canary parses, each so the gate can return a reason code carrying an artifact path instead of an anonymous parse failure. The constraint ledger carries both gaps.",
|
|
85
|
+
description: "One corpus probe. Succeeds the prior-art `h0-ground-truth` schema per AD-24, carrying its system identifier, implementation digest, `expectedClean` flag, seeded defects, and rationale, and adding AD-9's probe class and AD-9's per-probe artifact and commit digests. Divergences: `implementationSha` becomes `implementationDigest`, `taskId` does not survive because the probe pins what it describes by digest, and `expectedGate` does not survive because AD-40 makes detection a signature match rather than a verdict comparison and AD-7 keeps comparisons inside the dominance vector, so carrying an expected gate would invite a comparison the architecture forbids. Two constructions landed here together under one BREAKING `schemaVersion` bump: AD-9's per-class QUALIFICATION record, as a five-route tagged union required on every branch, and AD-40's machine-readable DEFECT SIGNATURE on the `expectedClean: false` branch, carrying the interface kind, the home operation as a method and a path template, the observable channel, and the discriminating condition. Both are required rather than optional, which is what makes the bump breaking under AD-11: every corpus written against version 1 fails to parse. Version 3 opened the signature to a second interface kind: it is a union on `interfaceKind`, where the `cli` branch declares a logical invocation in place of a method and a path template, and the selector's input binding carries the four command channels beside the four transport ones. What the schema still does not decide is stated rather than hidden. AD-9's \"an unqualified probe cannot enter a sealed set\" is a corpus-construction invariant enforced by the qualification gate in `core/score/qualification.ts`, not by this schema: all eight class-and-`expectedClean` pairings parse, a route incompatible with the pair parses, and a signature-less non-canary parses, each so the gate can return a reason code carrying an artifact path instead of an anonymous parse failure. The constraint ledger carries both gaps.",
|
|
86
86
|
});
|
|
@@ -5,8 +5,38 @@ import { JsonValue } from './primitives.ts';
|
|
|
5
5
|
* AD-23's verbatim quotation, paired with the channel it came from: a
|
|
6
6
|
* quotation with no channel cannot be audited against its source observation
|
|
7
7
|
* (ADR-009 Decision 2).
|
|
8
|
+
*
|
|
9
|
+
* Two arms rather than one shape with a nullable identifier and a `.refine()`
|
|
10
|
+
* over the pair. The identifier is meaningful on exactly one channel, and the
|
|
11
|
+
* first spelling of this field said so in prose while admitting every other
|
|
12
|
+
* combination: `channel: "response-body"` with a non-null `artifactId` parsed,
|
|
13
|
+
* and `projectChannel` reads the identifier only on the artifact channel, so
|
|
14
|
+
* the quotation was audited against the response body and a record naming a
|
|
15
|
+
* file it never consulted produced no condition at all. That is a narrower
|
|
16
|
+
* spelling of the wrong-file hole `artifactId` was added to close.
|
|
17
|
+
*
|
|
18
|
+
* A refinement would not have closed it either, for the reason
|
|
19
|
+
* `evidence-artifact.ts` records where it narrows a mode to a literal instead:
|
|
20
|
+
* a refinement never exports, so the published document keeps admitting the
|
|
21
|
+
* disagreement, and AD-13's corpus-mutation generator synthesises witnesses
|
|
22
|
+
* from a branch's own JSON Schema with no knowledge of a Zod-only cross-field
|
|
23
|
+
* rule. Narrowing the schema removes the gap at its source.
|
|
24
|
+
*
|
|
25
|
+
* A plain union rather than a discriminated one, on `DefectSignature`'s own
|
|
26
|
+
* reasoning: the discriminator would be `channel`, and one arm carries seven
|
|
27
|
+
* values for it. The two arms are told apart by the channel they name.
|
|
28
|
+
*
|
|
29
|
+
* The `EvidenceChannel` `$ref` is not carried at this site, since neither arm
|
|
30
|
+
* accepts the whole vocabulary. A non-TypeScript consumer reads a literal on
|
|
31
|
+
* one arm and a seven-member enum on the other, which is the rule itself
|
|
32
|
+
* rather than a reference plus a sentence about it.
|
|
8
33
|
*/
|
|
9
|
-
export declare const
|
|
34
|
+
export declare const ArtifactQuotedEvidence: z.ZodObject<{
|
|
35
|
+
quote: z.ZodString;
|
|
36
|
+
channel: z.ZodLiteral<"artifact">;
|
|
37
|
+
artifactId: z.ZodString;
|
|
38
|
+
}, z.core.$strict>;
|
|
39
|
+
export declare const ChannelQuotedEvidence: z.ZodObject<{
|
|
10
40
|
quote: z.ZodString;
|
|
11
41
|
channel: z.ZodEnum<{
|
|
12
42
|
"call-inputs": "call-inputs";
|
|
@@ -17,7 +47,25 @@ export declare const QuotedEvidence: z.ZodObject<{
|
|
|
17
47
|
stderr: "stderr";
|
|
18
48
|
stdout: "stdout";
|
|
19
49
|
}>;
|
|
50
|
+
artifactId: z.ZodNull;
|
|
20
51
|
}, z.core.$strict>;
|
|
52
|
+
export declare const QuotedEvidence: z.ZodUnion<readonly [z.ZodObject<{
|
|
53
|
+
quote: z.ZodString;
|
|
54
|
+
channel: z.ZodLiteral<"artifact">;
|
|
55
|
+
artifactId: z.ZodString;
|
|
56
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
57
|
+
quote: z.ZodString;
|
|
58
|
+
channel: z.ZodEnum<{
|
|
59
|
+
"call-inputs": "call-inputs";
|
|
60
|
+
"exit-code": "exit-code";
|
|
61
|
+
"response-body": "response-body";
|
|
62
|
+
"response-headers": "response-headers";
|
|
63
|
+
"response-status": "response-status";
|
|
64
|
+
stderr: "stderr";
|
|
65
|
+
stdout: "stdout";
|
|
66
|
+
}>;
|
|
67
|
+
artifactId: z.ZodNull;
|
|
68
|
+
}, z.core.$strict>]>;
|
|
21
69
|
/**
|
|
22
70
|
* `defect` findings additionally require observation identifiers and quoted
|
|
23
71
|
* evidence (AD-23). A nullable field can't express "required on this branch
|
|
@@ -49,7 +97,11 @@ export declare const Finding: z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
|
49
97
|
}, z.core.$strict>], "storage">>;
|
|
50
98
|
findingType: z.ZodLiteral<"defect">;
|
|
51
99
|
observationIds: z.ZodArray<z.ZodString>;
|
|
52
|
-
quotedEvidence: z.ZodArray<z.ZodObject<{
|
|
100
|
+
quotedEvidence: z.ZodArray<z.ZodUnion<readonly [z.ZodObject<{
|
|
101
|
+
quote: z.ZodString;
|
|
102
|
+
channel: z.ZodLiteral<"artifact">;
|
|
103
|
+
artifactId: z.ZodString;
|
|
104
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
53
105
|
quote: z.ZodString;
|
|
54
106
|
channel: z.ZodEnum<{
|
|
55
107
|
"call-inputs": "call-inputs";
|
|
@@ -60,7 +112,8 @@ export declare const Finding: z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
|
60
112
|
stderr: "stderr";
|
|
61
113
|
stdout: "stdout";
|
|
62
114
|
}>;
|
|
63
|
-
|
|
115
|
+
artifactId: z.ZodNull;
|
|
116
|
+
}, z.core.$strict>]>>;
|
|
64
117
|
}, z.core.$strict>, z.ZodObject<{
|
|
65
118
|
findingId: z.ZodString;
|
|
66
119
|
oracleId: z.ZodNullable<z.ZodString>;
|
|
@@ -147,6 +200,10 @@ export declare const ObservedCallInputs: z.ZodObject<{
|
|
|
147
200
|
query: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
148
201
|
header: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
149
202
|
body: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
203
|
+
argument: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
204
|
+
option: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
205
|
+
environment: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
206
|
+
stdin: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
150
207
|
}, z.core.$strict>;
|
|
151
208
|
/** the constraint identifier the ledger carries for the check below. */
|
|
152
209
|
export declare const OBSERVATION_SEQUENCE_UNIQUE = "observation-sequence-unique";
|
|
@@ -166,18 +223,48 @@ export declare const Observation: z.ZodObject<{
|
|
|
166
223
|
baseline: "baseline";
|
|
167
224
|
"evaluator-chosen": "evaluator-chosen";
|
|
168
225
|
}>;
|
|
226
|
+
principal: z.ZodNullable<z.ZodString>;
|
|
169
227
|
callInputs: z.ZodObject<{
|
|
170
228
|
path: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
171
229
|
query: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
172
230
|
header: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
173
231
|
body: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
232
|
+
argument: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
233
|
+
option: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
234
|
+
environment: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
235
|
+
stdin: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
174
236
|
}, z.core.$strict>;
|
|
175
237
|
responseBody: z.ZodNullable<z.ZodType<JsonValue, unknown, z.core.$ZodTypeInternals<JsonValue, unknown>>>;
|
|
176
238
|
responseHeaders: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
177
239
|
responseStatus: z.ZodNullable<z.ZodInt>;
|
|
178
|
-
stdout: z.
|
|
179
|
-
|
|
240
|
+
stdout: z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
241
|
+
kind: z.ZodLiteral<"json">;
|
|
242
|
+
value: z.ZodType<JsonValue, unknown, z.core.$ZodTypeInternals<JsonValue, unknown>>;
|
|
243
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
244
|
+
kind: z.ZodLiteral<"text">;
|
|
245
|
+
value: z.ZodString;
|
|
246
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
247
|
+
kind: z.ZodLiteral<"absent">;
|
|
248
|
+
}, z.core.$strict>], "kind">;
|
|
249
|
+
stderr: z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
250
|
+
kind: z.ZodLiteral<"json">;
|
|
251
|
+
value: z.ZodType<JsonValue, unknown, z.core.$ZodTypeInternals<JsonValue, unknown>>;
|
|
252
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
253
|
+
kind: z.ZodLiteral<"text">;
|
|
254
|
+
value: z.ZodString;
|
|
255
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
256
|
+
kind: z.ZodLiteral<"absent">;
|
|
257
|
+
}, z.core.$strict>], "kind">;
|
|
180
258
|
exitCode: z.ZodNullable<z.ZodInt>;
|
|
259
|
+
artifacts: z.ZodRecord<z.ZodString, z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
260
|
+
kind: z.ZodLiteral<"json">;
|
|
261
|
+
value: z.ZodType<JsonValue, unknown, z.core.$ZodTypeInternals<JsonValue, unknown>>;
|
|
262
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
263
|
+
kind: z.ZodLiteral<"text">;
|
|
264
|
+
value: z.ZodString;
|
|
265
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
266
|
+
kind: z.ZodLiteral<"absent">;
|
|
267
|
+
}, z.core.$strict>], "kind">>;
|
|
181
268
|
}, z.core.$strict>;
|
|
182
269
|
export type ObservedCallInputs = z.infer<typeof ObservedCallInputs>;
|
|
183
270
|
export type Observation = z.infer<typeof Observation>;
|
|
@@ -281,7 +368,11 @@ export declare const SealedRunRecord: z.ZodObject<{
|
|
|
281
368
|
}, z.core.$strict>], "storage">>;
|
|
282
369
|
findingType: z.ZodLiteral<"defect">;
|
|
283
370
|
observationIds: z.ZodArray<z.ZodString>;
|
|
284
|
-
quotedEvidence: z.ZodArray<z.ZodObject<{
|
|
371
|
+
quotedEvidence: z.ZodArray<z.ZodUnion<readonly [z.ZodObject<{
|
|
372
|
+
quote: z.ZodString;
|
|
373
|
+
channel: z.ZodLiteral<"artifact">;
|
|
374
|
+
artifactId: z.ZodString;
|
|
375
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
285
376
|
quote: z.ZodString;
|
|
286
377
|
channel: z.ZodEnum<{
|
|
287
378
|
"call-inputs": "call-inputs";
|
|
@@ -292,7 +383,8 @@ export declare const SealedRunRecord: z.ZodObject<{
|
|
|
292
383
|
stderr: "stderr";
|
|
293
384
|
stdout: "stdout";
|
|
294
385
|
}>;
|
|
295
|
-
|
|
386
|
+
artifactId: z.ZodNull;
|
|
387
|
+
}, z.core.$strict>]>>;
|
|
296
388
|
}, z.core.$strict>, z.ZodObject<{
|
|
297
389
|
findingId: z.ZodString;
|
|
298
390
|
oracleId: z.ZodNullable<z.ZodString>;
|
|
@@ -352,18 +444,48 @@ export declare const SealedRunRecord: z.ZodObject<{
|
|
|
352
444
|
baseline: "baseline";
|
|
353
445
|
"evaluator-chosen": "evaluator-chosen";
|
|
354
446
|
}>;
|
|
447
|
+
principal: z.ZodNullable<z.ZodString>;
|
|
355
448
|
callInputs: z.ZodObject<{
|
|
356
449
|
path: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
357
450
|
query: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
358
451
|
header: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
359
452
|
body: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
453
|
+
argument: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
454
|
+
option: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
455
|
+
environment: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
456
|
+
stdin: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
360
457
|
}, z.core.$strict>;
|
|
361
458
|
responseBody: z.ZodNullable<z.ZodType<JsonValue, unknown, z.core.$ZodTypeInternals<JsonValue, unknown>>>;
|
|
362
459
|
responseHeaders: z.ZodNullable<z.ZodType<import("./primitives.ts").JsonObject, unknown, z.core.$ZodTypeInternals<import("./primitives.ts").JsonObject, unknown>>>;
|
|
363
460
|
responseStatus: z.ZodNullable<z.ZodInt>;
|
|
364
|
-
stdout: z.
|
|
365
|
-
|
|
461
|
+
stdout: z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
462
|
+
kind: z.ZodLiteral<"json">;
|
|
463
|
+
value: z.ZodType<JsonValue, unknown, z.core.$ZodTypeInternals<JsonValue, unknown>>;
|
|
464
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
465
|
+
kind: z.ZodLiteral<"text">;
|
|
466
|
+
value: z.ZodString;
|
|
467
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
468
|
+
kind: z.ZodLiteral<"absent">;
|
|
469
|
+
}, z.core.$strict>], "kind">;
|
|
470
|
+
stderr: z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
471
|
+
kind: z.ZodLiteral<"json">;
|
|
472
|
+
value: z.ZodType<JsonValue, unknown, z.core.$ZodTypeInternals<JsonValue, unknown>>;
|
|
473
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
474
|
+
kind: z.ZodLiteral<"text">;
|
|
475
|
+
value: z.ZodString;
|
|
476
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
477
|
+
kind: z.ZodLiteral<"absent">;
|
|
478
|
+
}, z.core.$strict>], "kind">;
|
|
366
479
|
exitCode: z.ZodNullable<z.ZodInt>;
|
|
480
|
+
artifacts: z.ZodRecord<z.ZodString, z.ZodDiscriminatedUnion<[z.ZodObject<{
|
|
481
|
+
kind: z.ZodLiteral<"json">;
|
|
482
|
+
value: z.ZodType<JsonValue, unknown, z.core.$ZodTypeInternals<JsonValue, unknown>>;
|
|
483
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
484
|
+
kind: z.ZodLiteral<"text">;
|
|
485
|
+
value: z.ZodString;
|
|
486
|
+
}, z.core.$strict>, z.ZodObject<{
|
|
487
|
+
kind: z.ZodLiteral<"absent">;
|
|
488
|
+
}, z.core.$strict>], "kind">>;
|
|
367
489
|
}, z.core.$strict>>;
|
|
368
490
|
judgeResults: z.ZodArray<z.ZodObject<{
|
|
369
491
|
rubricId: z.ZodString;
|
|
@@ -3,21 +3,58 @@ import { z } from 'zod';
|
|
|
3
3
|
import { ArtifactReference } from './artifact-reference.js';
|
|
4
4
|
import { Severity } from './eval-contract.js';
|
|
5
5
|
import { lineageFields } from './lineage.js';
|
|
6
|
-
import {
|
|
6
|
+
import { IDENTIFIER_ROOTED_CHANNEL, NON_IDENTIFIER_ROOTED_CHANNELS, } from './pointer.js';
|
|
7
7
|
import { BehaviorId, Digest, FindingId, Identifier, JsonObjectValue, JsonValue, OracleId, ProbeId, RubricCriterionId, RubricId, UnsignedDecimalString, } from './primitives.js';
|
|
8
|
+
import { ProbeObservedBody } from './probe-body.js';
|
|
8
9
|
import { EvaluatorRecommendation } from './verdict.js';
|
|
10
|
+
const QUOTE_DESCRIPTION = "The evaluator's verbatim quotation, per AD-23. Non-empty: an empty quotation quotes nothing, no AD-5 code names the condition, and under the admit-rule's second clause the schema is therefore the enforcement point. That this text appears in at least one of the finding's cited observations is NOT checked here; it is an AD-32 declared-versus-observed inconsistency that invalidates at ingest, and ADR-009 Decision 2 settles the precedence: \"cited identifiers govern the witness match; quotation audits it.\"";
|
|
11
|
+
const quote = z.string().min(1).describe(QUOTE_DESCRIPTION);
|
|
9
12
|
/**
|
|
10
13
|
* AD-23's verbatim quotation, paired with the channel it came from: a
|
|
11
14
|
* quotation with no channel cannot be audited against its source observation
|
|
12
15
|
* (ADR-009 Decision 2).
|
|
16
|
+
*
|
|
17
|
+
* Two arms rather than one shape with a nullable identifier and a `.refine()`
|
|
18
|
+
* over the pair. The identifier is meaningful on exactly one channel, and the
|
|
19
|
+
* first spelling of this field said so in prose while admitting every other
|
|
20
|
+
* combination: `channel: "response-body"` with a non-null `artifactId` parsed,
|
|
21
|
+
* and `projectChannel` reads the identifier only on the artifact channel, so
|
|
22
|
+
* the quotation was audited against the response body and a record naming a
|
|
23
|
+
* file it never consulted produced no condition at all. That is a narrower
|
|
24
|
+
* spelling of the wrong-file hole `artifactId` was added to close.
|
|
25
|
+
*
|
|
26
|
+
* A refinement would not have closed it either, for the reason
|
|
27
|
+
* `evidence-artifact.ts` records where it narrows a mode to a literal instead:
|
|
28
|
+
* a refinement never exports, so the published document keeps admitting the
|
|
29
|
+
* disagreement, and AD-13's corpus-mutation generator synthesises witnesses
|
|
30
|
+
* from a branch's own JSON Schema with no knowledge of a Zod-only cross-field
|
|
31
|
+
* rule. Narrowing the schema removes the gap at its source.
|
|
32
|
+
*
|
|
33
|
+
* A plain union rather than a discriminated one, on `DefectSignature`'s own
|
|
34
|
+
* reasoning: the discriminator would be `channel`, and one arm carries seven
|
|
35
|
+
* values for it. The two arms are told apart by the channel they name.
|
|
36
|
+
*
|
|
37
|
+
* The `EvidenceChannel` `$ref` is not carried at this site, since neither arm
|
|
38
|
+
* accepts the whole vocabulary. A non-TypeScript consumer reads a literal on
|
|
39
|
+
* one arm and a seven-member enum on the other, which is the rule itself
|
|
40
|
+
* rather than a reference plus a sentence about it.
|
|
13
41
|
*/
|
|
14
|
-
export const
|
|
15
|
-
quote
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
42
|
+
export const ArtifactQuotedEvidence = z.strictObject({
|
|
43
|
+
quote,
|
|
44
|
+
channel: z.literal(IDENTIFIER_ROOTED_CHANNEL),
|
|
45
|
+
artifactId: Identifier.describe("Which written file the quotation came from. A channel alone does not identify one: an operation may write several, and an audit that searched all of them at once accepted a quotation lifted from a file the finding never cited, which is the opposite of what AD-23's verbatim requirement is for."),
|
|
46
|
+
});
|
|
47
|
+
export const ChannelQuotedEvidence = z.strictObject({
|
|
48
|
+
quote,
|
|
49
|
+
channel: z.enum(NON_IDENTIFIER_ROOTED_CHANNELS),
|
|
50
|
+
artifactId: z
|
|
51
|
+
.null()
|
|
52
|
+
.describe('Null on every channel but `artifact`, and required rather than optional so a record cannot omit the question. Only a written file needs naming; every other channel is one thing per observation.'),
|
|
20
53
|
});
|
|
54
|
+
export const QuotedEvidence = z.union([
|
|
55
|
+
ArtifactQuotedEvidence,
|
|
56
|
+
ChannelQuotedEvidence,
|
|
57
|
+
]);
|
|
21
58
|
// Spread into each finding branch rather than shared as a base object: a
|
|
22
59
|
// spread adds no `$defs` entry, so each branch exports as a complete shape a
|
|
23
60
|
// non-TypeScript consumer can read without following a reference (AD-13).
|
|
@@ -108,6 +145,10 @@ export const ObservedCallInputs = z.strictObject({
|
|
|
108
145
|
query: JsonObjectValue.nullable(),
|
|
109
146
|
header: JsonObjectValue.nullable(),
|
|
110
147
|
body: JsonObjectValue.nullable(),
|
|
148
|
+
argument: JsonObjectValue.nullable(),
|
|
149
|
+
option: JsonObjectValue.nullable(),
|
|
150
|
+
environment: JsonObjectValue.nullable(),
|
|
151
|
+
stdin: JsonObjectValue.nullable(),
|
|
111
152
|
});
|
|
112
153
|
/** the constraint identifier the ledger carries for the check below. */
|
|
113
154
|
export const OBSERVATION_SEQUENCE_UNIQUE = 'observation-sequence-unique';
|
|
@@ -129,6 +170,7 @@ export const Observation = z.strictObject({
|
|
|
129
170
|
provenance: z
|
|
130
171
|
.enum(['baseline', 'evaluator-chosen'])
|
|
131
172
|
.describe("AD-23: `baseline` for a pre-canned or deterministic test, `evaluator-chosen` for an action the evaluator selected. The distinction is the one the product's central finding rests on: what a sealed evaluator detects beyond the pre-canned baseline. It lives on the observation, never on the finding."),
|
|
173
|
+
principal: Identifier.nullable().describe("The declared principal the harness acted as, or `null` where the run named none. Owed item 3's other half: a `{ principal }` input binding is presence-only by construction, since the contract declares a name and the harness provisions the value, so without this field two steps of one operation binding `owner` and `other-user` both match every observation and both resolve `several`. That is exactly the act-as-A-read-as-B shape the two critical-severity cross-user behaviours need, and it was unscoreable while the record said nothing about which account was used. An opaque label carrying no account identifier or credential, on `testData.principals`' own AD-18 terms."),
|
|
132
174
|
callInputs: ObservedCallInputs,
|
|
133
175
|
responseBody: JsonValue.nullable().describe('AD-26\'s `response-body` channel. The null branch is redundant against the value container, which already admits `null`; it is kept so all ten observation fields read the same way, and it means "no body observed" and "a body that was JSON null" are indistinguishable here, which is an accepted cost of one uniform spelling.'),
|
|
134
176
|
responseHeaders: JsonObjectValue.nullable().describe('A name-to-value map, not the open value container. AD-26 gives `response-headers` a tail, so a pointer resolves INTO this channel; a scalar here would leave `/interactions/x/response-headers/Content-Type` addressing nothing. That is the difference from `responseBody`, where a scalar or an array is a legitimate body and the open container is correct.'),
|
|
@@ -137,12 +179,15 @@ export const Observation = z.strictObject({
|
|
|
137
179
|
.min(0)
|
|
138
180
|
.nullable()
|
|
139
181
|
.describe('Deliberately not bounded to a protocol range. A negative status is meaningless and excluded, but the upper end is left open: AD-19 declares four interface kinds and v0 rejects three of them at compile time under `unsupported-interface-kind`, so bounding this to HTTP would encode a protocol assumption the artifact outlives. `null` where the channel does not apply.'),
|
|
140
|
-
stdout:
|
|
141
|
-
stderr:
|
|
182
|
+
stdout: ProbeObservedBody.describe('AD-26\'s `stdout` channel, tagged rather than a bare string. An operation may nominate standard output as the channel its response descriptor describes, in which case a pointer descends into it, and an untagged string could not tell output that was JSON from output that was only ever text. `{ "kind": "absent" }` is the channel a run did not observe.'),
|
|
183
|
+
stderr: ProbeObservedBody.describe("AD-26's `stderr` channel, tagged on the same terms as `stdout`."),
|
|
142
184
|
exitCode: z
|
|
143
185
|
.int()
|
|
144
186
|
.nullable()
|
|
145
187
|
.describe('Signed on purpose, unlike `responseStatus`: a process terminated by a signal is conventionally reported as a negative code, and this field records what was observed rather than what is tidy.'),
|
|
188
|
+
artifacts: z
|
|
189
|
+
.record(Identifier, ProbeObservedBody)
|
|
190
|
+
.describe("AD-26's `artifact` channel, keyed by the identifier the operation declares it writes, which is the segment an artifact pointer carries before its tail. Caller-keyed and expected to be partial: a run that wrote no files carries `{}`. Tagged on the same terms as `stdout`, since a written report may be JSON, may be prose, and may not have been written at all."),
|
|
146
191
|
});
|
|
147
192
|
/**
|
|
148
193
|
* AD-17: judge results arrive inside the sealed run record and the package
|
|
@@ -154,7 +199,7 @@ export const JudgeResult = z.strictObject({
|
|
|
154
199
|
score: z
|
|
155
200
|
.int()
|
|
156
201
|
.nullable()
|
|
157
|
-
.describe("An integer, because AD-22 puts the scale on the rubric's own anchored levels and `ScaleLevel.level` is already `z.int()`; no second scale is minted here. `null` is the shape AD-6's `judge-error` fires on, so it must parse. That the criterion is one the cited rubric
|
|
202
|
+
.describe("An integer, because AD-22 puts the scale on the rubric's own anchored levels and `ScaleLevel.level` is already `z.int()`; no second scale is minted here. `null` is the shape AD-6's `judge-error` fires on, so it must parse. Two AD-17 rules sit over this field and only one of them is decidable from a record. That a criterion is scored at most once IS: `core/ingest` reports a repeated `rubricId`/`criterionId` pair as a `duplicate-record-identifier` condition, since a criterion scored twice is not one judge call's product whatever else the record says. That a scored criterion is one the cited rubric DECLARES is not, and never will be from inside this package: the rubric is not among ingest's declared inputs, no stage row names it, and a schema over one artifact has no second operand. That half is the caller's, on the same terms AD-12 states for the remediation cap, and it is a stated boundary rather than an unimplemented rule."),
|
|
158
203
|
note: z.string().nullable(),
|
|
159
204
|
});
|
|
160
205
|
/** the prior art's five members, with money as a string per AD-36. */
|
|
@@ -243,5 +288,5 @@ export const SealedRunRecord = z
|
|
|
243
288
|
})
|
|
244
289
|
.meta({
|
|
245
290
|
id: 'SealedRunRecord',
|
|
246
|
-
description: "One sealed evaluator trial, as the caller presents it. Succeeds the prior-art `h0-run-result` schema per AD-24, keeping its run identifier, condition arm, findings, action-log reference, resource use, invalidation reason, evaluator recommendation as a closed enum, and per-finding confidence on a declared scale. Divergences: `condition` is demoted to the opaque `conditionArm`, `verdict` becomes `evaluatorRecommendation` without `NOT_APPLICABLE`, money is a decimal string, and `taskId`, `note`, and per-finding `actionIds` do not survive: the contract is pinned by `contractDigest`, an unstructured orchestrator annotation is the free-prose channel the Conventions close everywhere else, and two citation vocabularies on one finding is the ambiguity ADR-009 removed. The run MODE landed here as a required field under a BREAKING `schemaVersion` bump, which is where AD-21's \"fixed before ingest\" puts it; owed item 4 is now closed: mode enters AD-11's identity inputs as `ScoringVersionInputs`'s sixth field, and `core/score/ladder.ts` carries `ProductionAssessment`/`ContractAssessment` as the two assessment input types with their own total ladders. Observation ORDERING landed here too, under its own BREAKING `schemaVersion` bump: `sequence` is required and unique per record, closing owed item 2's ADR-006 gap, since array position was never a legal ordering.",
|
|
291
|
+
description: "One sealed evaluator trial, as the caller presents it. Succeeds the prior-art `h0-run-result` schema per AD-24, keeping its run identifier, condition arm, findings, action-log reference, resource use, invalidation reason, evaluator recommendation as a closed enum, and per-finding confidence on a declared scale. Divergences: `condition` is demoted to the opaque `conditionArm`, `verdict` becomes `evaluatorRecommendation` without `NOT_APPLICABLE`, money is a decimal string, and `taskId`, `note`, and per-finding `actionIds` do not survive: the contract is pinned by `contractDigest`, an unstructured orchestrator annotation is the free-prose channel the Conventions close everywhere else, and two citation vocabularies on one finding is the ambiguity ADR-009 removed. The run MODE landed here as a required field under a BREAKING `schemaVersion` bump, which is where AD-21's \"fixed before ingest\" puts it; owed item 4 is now closed: mode enters AD-11's identity inputs as `ScoringVersionInputs`'s sixth field, and `core/score/ladder.ts` carries `ProductionAssessment`/`ContractAssessment` as the two assessment input types with their own total ladders. Observation ORDERING landed here too, under its own BREAKING `schemaVersion` bump: `sequence` is required and unique per record, closing owed item 2's ADR-006 gap, since array position was never a legal ordering. Version 4 opened the record to a system under test that runs behind a command: `callInputs` carries the four command channels beside the four transport ones, `stdout` and `stderr` are tagged rather than bare strings so a nominated output channel can be descended into, and `artifacts` records the files the run wrote, keyed by the identifier the operation declares.",
|
|
247
292
|
});
|