modelrig 0.9.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +179 -0
- package/README.md +51 -27
- package/data/PROVENANCE.md +14 -0
- package/data/pricing-map.json +26 -0
- package/dist/adapters/attempt-meta-map.d.ts +8 -1
- package/dist/adapters/attempt-meta-map.d.ts.map +1 -1
- package/dist/adapters/attempt-meta-map.js +4 -0
- package/dist/adapters/build-adapters.js +2 -2
- package/dist/adapters/gemini.d.ts +28 -1
- package/dist/adapters/gemini.d.ts.map +1 -1
- package/dist/adapters/gemini.js +38 -9
- package/dist/adapters/grok.d.ts.map +1 -1
- package/dist/adapters/grok.js +53 -4
- package/dist/adapters/openai-compat.d.ts +8 -2
- package/dist/adapters/openai-compat.d.ts.map +1 -1
- package/dist/adapters/openai-compat.js +8 -1
- package/dist/adapters/openai-stream.d.ts +10 -2
- package/dist/adapters/openai-stream.d.ts.map +1 -1
- package/dist/adapters/openai-stream.js +5 -1
- package/dist/adapters/openai.d.ts +10 -1
- package/dist/adapters/openai.d.ts.map +1 -1
- package/dist/adapters/openai.js +57 -7
- package/dist/adapters/responses-api.d.ts +61 -0
- package/dist/adapters/responses-api.d.ts.map +1 -0
- package/dist/adapters/responses-api.js +124 -0
- package/dist/artifacts/artifact.d.ts +20 -0
- package/dist/artifacts/artifact.d.ts.map +1 -1
- package/dist/artifacts/artifact.js +207 -171
- package/dist/artifacts/index.d.ts +3 -1
- package/dist/artifacts/index.d.ts.map +1 -1
- package/dist/artifacts/index.js +8 -1
- package/dist/bakeoff/groundedness.d.ts +58 -0
- package/dist/bakeoff/groundedness.d.ts.map +1 -0
- package/dist/bakeoff/groundedness.js +165 -0
- package/dist/bakeoff/quality.d.ts +14 -1
- package/dist/bakeoff/quality.d.ts.map +1 -1
- package/dist/bakeoff/quality.js +10 -0
- package/dist/bakeoff/replay.d.ts.map +1 -1
- package/dist/bakeoff/replay.js +4 -0
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +8 -0
- package/dist/config.d.ts +31 -0
- package/dist/config.d.ts.map +1 -1
- package/dist/config.js +23 -0
- package/dist/doctor-cli.d.ts +7 -0
- package/dist/doctor-cli.d.ts.map +1 -0
- package/dist/doctor-cli.js +67 -0
- package/dist/doctor-preflight.d.ts +52 -0
- package/dist/doctor-preflight.d.ts.map +1 -0
- package/dist/doctor-preflight.js +112 -0
- package/dist/feedback/grade.d.ts +8 -1
- package/dist/feedback/grade.d.ts.map +1 -1
- package/dist/feedback/grade.js +4 -0
- package/dist/gateway/http-protocol.d.ts +19 -0
- package/dist/gateway/http-protocol.d.ts.map +1 -1
- package/dist/gateway/serve-client.d.ts +99 -0
- package/dist/gateway/serve-client.d.ts.map +1 -0
- package/dist/gateway/serve-client.js +360 -0
- package/dist/index.d.ts +25 -3
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +62 -8
- package/dist/prompts/manifest.d.ts +128 -0
- package/dist/prompts/manifest.d.ts.map +1 -0
- package/dist/prompts/manifest.js +214 -0
- package/dist/prompts/prompt-fetch.d.ts +84 -0
- package/dist/prompts/prompt-fetch.d.ts.map +1 -0
- package/dist/prompts/prompt-fetch.js +120 -0
- package/dist/prompts/prompt-resolve.d.ts +76 -0
- package/dist/prompts/prompt-resolve.d.ts.map +1 -0
- package/dist/prompts/prompt-resolve.js +93 -0
- package/dist/prompts/slot-resolver.d.ts +91 -0
- package/dist/prompts/slot-resolver.d.ts.map +1 -0
- package/dist/prompts/slot-resolver.js +135 -0
- package/dist/prompts/tools-register.d.ts +60 -0
- package/dist/prompts/tools-register.d.ts.map +1 -0
- package/dist/prompts/tools-register.js +48 -0
- package/dist/prompts/version-hash.d.ts +29 -0
- package/dist/prompts/version-hash.d.ts.map +1 -0
- package/dist/prompts/version-hash.js +54 -0
- package/dist/prompts-export-cli.d.ts +21 -0
- package/dist/prompts-export-cli.d.ts.map +1 -0
- package/dist/prompts-export-cli.js +130 -0
- package/dist/registry/cache-policy.d.ts.map +1 -1
- package/dist/registry/cache-policy.js +22 -0
- package/dist/registry/pricing-shape.d.ts +38 -8
- package/dist/registry/pricing-shape.d.ts.map +1 -1
- package/dist/registry/pricing-shape.js +40 -12
- package/dist/registry/pricing.d.ts +6 -5
- package/dist/registry/pricing.d.ts.map +1 -1
- package/dist/registry/pricing.js +9 -5
- package/dist/routes/destination-loader.d.ts +16 -0
- package/dist/routes/destination-loader.d.ts.map +1 -0
- package/dist/routes/destination-loader.js +29 -0
- package/dist/routes/draft.d.ts +7 -0
- package/dist/routes/draft.d.ts.map +1 -1
- package/dist/routes/draft.js +12 -0
- package/dist/routes/loader.d.ts.map +1 -1
- package/dist/routes/loader.js +27 -54
- package/dist/routes/output-loader.d.ts +42 -0
- package/dist/routes/output-loader.d.ts.map +1 -0
- package/dist/routes/output-loader.js +112 -0
- package/dist/routes/prompt-loader.d.ts +49 -0
- package/dist/routes/prompt-loader.d.ts.map +1 -0
- package/dist/routes/prompt-loader.js +113 -0
- package/dist/routes/resolved-bundle.d.ts +19 -0
- package/dist/routes/resolved-bundle.d.ts.map +1 -1
- package/dist/routes/resolved-bundle.js +24 -0
- package/dist/routes/screen-loader.d.ts +1 -1
- package/dist/routes/screen-loader.d.ts.map +1 -1
- package/dist/routes/screen-loader.js +8 -1
- package/dist/run/accept.d.ts +3 -0
- package/dist/run/accept.d.ts.map +1 -1
- package/dist/run/accept.js +1 -0
- package/dist/run/cost.d.ts +31 -2
- package/dist/run/cost.d.ts.map +1 -1
- package/dist/run/cost.js +57 -2
- package/dist/run/destination.d.ts +39 -0
- package/dist/run/destination.d.ts.map +1 -0
- package/dist/run/destination.js +45 -0
- package/dist/run/dispatch.d.ts +25 -3
- package/dist/run/dispatch.d.ts.map +1 -1
- package/dist/run/dispatch.js +26 -4
- package/dist/run/raw-defaults.d.ts.map +1 -1
- package/dist/run/raw-defaults.js +6 -1
- package/dist/run/raw-finalize.d.ts +50 -0
- package/dist/run/raw-finalize.d.ts.map +1 -0
- package/dist/run/raw-finalize.js +266 -0
- package/dist/run/raw-knobs.d.ts +36 -0
- package/dist/run/raw-knobs.d.ts.map +1 -0
- package/dist/run/raw-knobs.js +99 -0
- package/dist/run/raw-request.d.ts +24 -0
- package/dist/run/raw-request.d.ts.map +1 -0
- package/dist/run/raw-request.js +69 -0
- package/dist/run/raw-schema.d.ts +76 -0
- package/dist/run/raw-schema.d.ts.map +1 -0
- package/dist/run/raw-schema.js +166 -0
- package/dist/run/raw-search-cost.d.ts +48 -0
- package/dist/run/raw-search-cost.d.ts.map +1 -0
- package/dist/run/raw-search-cost.js +78 -0
- package/dist/run/raw-support.d.ts +64 -6
- package/dist/run/raw-support.d.ts.map +1 -1
- package/dist/run/raw-support.js +84 -11
- package/dist/run/raw.d.ts +25 -26
- package/dist/run/raw.d.ts.map +1 -1
- package/dist/run/raw.js +30 -351
- package/dist/run/record-attempt.d.ts +8 -0
- package/dist/run/record-attempt.d.ts.map +1 -1
- package/dist/run/record-attempt.js +26 -12
- package/dist/run/repair-attempt.d.ts +68 -0
- package/dist/run/repair-attempt.d.ts.map +1 -0
- package/dist/run/repair-attempt.js +180 -0
- package/dist/run/resolve-destination.d.ts +49 -0
- package/dist/run/resolve-destination.d.ts.map +1 -0
- package/dist/run/resolve-destination.js +76 -0
- package/dist/run/routing-docs.d.ts.map +1 -1
- package/dist/run/routing-docs.js +9 -4
- package/dist/run/run.d.ts +30 -1
- package/dist/run/run.d.ts.map +1 -1
- package/dist/run/run.js +75 -148
- package/dist/run/telemetry-row.d.ts +92 -0
- package/dist/run/telemetry-row.d.ts.map +1 -0
- package/dist/run/telemetry-row.js +81 -0
- package/dist/run/validate.d.ts +12 -1
- package/dist/run/validate.d.ts.map +1 -1
- package/dist/run/validate.js +31 -0
- package/dist/run/web-search-pricing.d.ts +43 -0
- package/dist/run/web-search-pricing.d.ts.map +1 -0
- package/dist/run/web-search-pricing.js +56 -0
- package/dist/settings/grading-guide.d.ts +27 -1
- package/dist/settings/grading-guide.d.ts.map +1 -1
- package/dist/status.d.ts +6 -0
- package/dist/status.d.ts.map +1 -1
- package/dist/status.js +4 -1
- package/dist/telemetry/artifact-store.d.ts.map +1 -1
- package/dist/telemetry/artifact-store.js +11 -0
- package/dist/telemetry/attempt-meta.d.ts +55 -1
- package/dist/telemetry/attempt-meta.d.ts.map +1 -1
- package/dist/telemetry/attempt-meta.js +24 -1
- package/dist/telemetry/exporter-setup.d.ts +2 -0
- package/dist/telemetry/exporter-setup.d.ts.map +1 -1
- package/dist/telemetry/exporter-setup.js +6 -1
- package/dist/telemetry/grade-mappers.d.ts +5 -2
- package/dist/telemetry/grade-mappers.d.ts.map +1 -1
- package/dist/telemetry/grade-mappers.js +8 -2
- package/dist/telemetry/grades-sql.d.ts +4 -1
- package/dist/telemetry/grades-sql.d.ts.map +1 -1
- package/dist/telemetry/grades-sql.js +26 -3
- package/dist/telemetry/ingest-protocol.d.ts +26 -2
- package/dist/telemetry/ingest-protocol.d.ts.map +1 -1
- package/dist/telemetry/ingest-protocol.js +32 -29
- package/dist/telemetry/org-identity.d.ts +44 -0
- package/dist/telemetry/org-identity.d.ts.map +1 -0
- package/dist/telemetry/org-identity.js +98 -0
- package/dist/telemetry/route-registration.d.ts +48 -0
- package/dist/telemetry/route-registration.d.ts.map +1 -0
- package/dist/telemetry/route-registration.js +77 -0
- package/dist/telemetry/sinks.d.ts +10 -1
- package/dist/telemetry/sinks.d.ts.map +1 -1
- package/dist/telemetry/sinks.js +52 -6
- package/dist/telemetry/store-migrations.d.ts.map +1 -1
- package/dist/telemetry/store-migrations.js +13 -0
- package/dist/tenancy/api-key-format.d.ts +16 -1
- package/dist/tenancy/api-key-format.d.ts.map +1 -1
- package/dist/tenancy/api-key-format.js +17 -0
- package/dist/types/provider.d.ts +20 -3
- package/dist/types/provider.d.ts.map +1 -1
- package/dist/types/provider.js +1 -0
- package/dist/types/route.d.ts +76 -0
- package/dist/types/route.d.ts.map +1 -1
- package/dist/types/telemetry.d.ts +30 -2
- package/dist/types/telemetry.d.ts.map +1 -1
- package/docs/agent-skill/SKILL.md +73 -5
- package/docs/instrumentation.md +81 -36
- package/docs/llms.txt +95 -16
- package/docs/managed-serving.md +150 -0
- package/docs/mcp-oracle.md +75 -2
- package/docs/migration-playbook.md +5 -4
- package/docs/quickstart.md +169 -83
- package/package.json +3 -2
- package/registry/registry.json +82 -0
- package/dist/feedback/feedback.d.ts +0 -35
- package/dist/feedback/feedback.d.ts.map +0 -1
- package/dist/feedback/feedback.js +0 -39
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"api-key-format.d.ts","sourceRoot":"","sources":["../../src/tenancy/api-key-format.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;GAaG;AAIH;;;GAGG;AACH,eAAO,MAAM,cAAc,YAAY,CAAC;AAExC;;6EAE6E;AAC7E,eAAO,MAAM,oBAAoB,KAAK,CAAC;AAEvC,oFAAoF;AACpF,eAAO,MAAM,uBAAuB,KAAK,CAAC;AAE1C
|
|
1
|
+
{"version":3,"file":"api-key-format.d.ts","sourceRoot":"","sources":["../../src/tenancy/api-key-format.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;GAaG;AAIH;;;GAGG;AACH,eAAO,MAAM,cAAc,YAAY,CAAC;AAExC;;6EAE6E;AAC7E,eAAO,MAAM,oBAAoB,KAAK,CAAC;AAEvC,oFAAoF;AACpF,eAAO,MAAM,uBAAuB,KAAK,CAAC;AAE1C;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GA2DG;AACH,eAAO,MAAM,cAAc,iMAcjB,CAAC;AAEX,MAAM,MAAM,WAAW,GAAG,CAAC,OAAO,cAAc,CAAC,CAAC,MAAM,CAAC,CAAC;AAE1D;mDACmD;AACnD,wBAAgB,gBAAgB,CAAC,MAAM,EAAE,MAAM,GAAG,MAAM,CAEvD;AAED;;;;;;;GAOG;AACH,wBAAgB,eAAe,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAI1D;AAED;;;;;;GAMG;AACH,wBAAgB,YAAY,CAAC,SAAS,EAAE,MAAM,GAAG,MAAM,CAGtD;AAED;;;;;;;;GAQG;AACH,wBAAgB,WAAW,CAAC,GAAG,EAAE,OAAO,GAAG,WAAW,EAAE,CAIvD;AAED;gEACgE;AAChE,wBAAgB,QAAQ,CAAC,OAAO,EAAE,SAAS,WAAW,EAAE,EAAE,QAAQ,EAAE,WAAW,GAAG,OAAO,CAExF"}
|
|
@@ -62,6 +62,21 @@ exports.API_KEY_BODY_MIN_LENGTH = 43;
|
|
|
62
62
|
* settings needs no scope — any org-scoped key may read, since they ride the
|
|
63
63
|
* already-readable orgs row.
|
|
64
64
|
*
|
|
65
|
+
* `serve` (managed-serving-keyless K2, 00084): SERVE a managed inference on the
|
|
66
|
+
* hosted gateway's MANAGED mirror lane (`POST /v1/chat/completions` resolving an
|
|
67
|
+
* org's mirrored route against OUR managed provider keys). This is the first-class
|
|
68
|
+
* inference authority the mirror lane was always meant to gate on — it replaces the
|
|
69
|
+
* overloaded `oracle-act` placeholder there (gateway-routes.ts flagged that as an
|
|
70
|
+
* interim stand-in for a real inference/serve scope). It is an ORDINARY machine
|
|
71
|
+
* grant (not owner-only to mint): serving is a delegated coding-agent act, like
|
|
72
|
+
* `route-config`/`prompt-write`, not a money/deletion authority. It is DISTINCT
|
|
73
|
+
* from per-org PROVISIONING: a key may carry `serve` and still be refused because
|
|
74
|
+
* the org is not provisioned for managed serving (orgs.managed_serving_enabled,
|
|
75
|
+
* 00085) — the scope says "this key may serve", the column says "this org may".
|
|
76
|
+
* During the K2 deprecation window the mirror lane still ALSO accepts `oracle-act`
|
|
77
|
+
* so keys minted before the remint keep working; that acceptance is removed once
|
|
78
|
+
* keys are reminted with `serve`.
|
|
79
|
+
*
|
|
65
80
|
* `artifact-delete` (CD-1 / R-7, 00068): DELETE an org's own stored artifact
|
|
66
81
|
* CONTENT (`DELETE /v1/artifacts/:id/content`) or PURGE the record
|
|
67
82
|
* (`DELETE /v1/artifacts/:id`). Held by an automation deleting on the org's
|
|
@@ -89,6 +104,8 @@ exports.API_KEY_SCOPES = [
|
|
|
89
104
|
"route-config",
|
|
90
105
|
"org-settings",
|
|
91
106
|
"artifact-delete",
|
|
107
|
+
"prompt-write",
|
|
108
|
+
"serve",
|
|
92
109
|
];
|
|
93
110
|
/** sha256 of the WHOLE presented string, prefix included — so a caller cannot
|
|
94
111
|
* strip the prefix and land on a different row. */
|
package/dist/types/provider.d.ts
CHANGED
|
@@ -5,13 +5,13 @@
|
|
|
5
5
|
*/
|
|
6
6
|
import type { RigFailure } from "./failures";
|
|
7
7
|
import type { AttemptMetaV1 } from "../telemetry/attempt-meta";
|
|
8
|
-
export type ProviderId = "gemini" | "openai" | "deepseek" | "anthropic" | "grok" | "deepinfra" | "fireworks";
|
|
8
|
+
export type ProviderId = "gemini" | "openai" | "deepseek" | "anthropic" | "grok" | "deepinfra" | "fireworks" | "zai";
|
|
9
9
|
/** Runtime enumeration of every ProviderId — the single list a raw-model lane
|
|
10
10
|
* (`provider/model`) validates against (http gateway spec §3.3). Kept in sync
|
|
11
11
|
* with the `ProviderId` union by the two compile-time assertions below: adding
|
|
12
12
|
* a provider to the union without adding it here (or vice-versa) fails the
|
|
13
13
|
* type-check. */
|
|
14
|
-
export declare const PROVIDER_IDS: readonly ["gemini", "openai", "deepseek", "anthropic", "grok", "deepinfra", "fireworks"];
|
|
14
|
+
export declare const PROVIDER_IDS: readonly ["gemini", "openai", "deepseek", "anthropic", "grok", "deepinfra", "fireworks", "zai"];
|
|
15
15
|
export type CapabilityFlag = "structured_native" | "json_mode" | "grounded_native" | "context_cache_explicit" | "prompt_cache_key" | "prefix_cache_implicit" | "tier_flex" | "trace_visible" | "zero_retention";
|
|
16
16
|
/** The canonical flag enumeration — the single list every static capability
|
|
17
17
|
* materialization iterates (index.ts, template validation, tests). */
|
|
@@ -112,7 +112,9 @@ export interface AdapterRequest {
|
|
|
112
112
|
readonly candidate: CandidateRef;
|
|
113
113
|
readonly systemPrompt: string;
|
|
114
114
|
readonly userPrompt: string;
|
|
115
|
-
/** JSON Schema for native structured output; null => prompt/json_mode path.
|
|
115
|
+
/** JSON Schema for native structured output; null => prompt/json_mode path.
|
|
116
|
+
* Set by the routed lane from the bundle's `schema`, and by the raw lane from
|
|
117
|
+
* a declared `RawRunInput.responseSchema` (R2) — the SAME adapter path. */
|
|
116
118
|
readonly outputSchema: object | null;
|
|
117
119
|
readonly requestedTier: ServiceTier;
|
|
118
120
|
readonly timeoutMs: number;
|
|
@@ -214,6 +216,21 @@ export type AdapterResult = {
|
|
|
214
216
|
* A tool-call turn carries these instead of (or alongside) `text`; the
|
|
215
217
|
* run loop surfaces them and does NOT schema-validate the turn. */
|
|
216
218
|
readonly toolCalls?: readonly ToolCall[];
|
|
219
|
+
/** Native web-grounding citations (Native Web Search lane, spec N2) —
|
|
220
|
+
* present only on a grounded Responses-API attempt (openai/grok), in the
|
|
221
|
+
* inject rung's `url_citation` shape (run/grounding.ts). ADDITIVE: absent on
|
|
222
|
+
* every non-grounded call, so nothing else changes. The raw lane surfaces
|
|
223
|
+
* these on `RunMeta.citations`; a zero-citation grounded response carries an
|
|
224
|
+
* empty array (surfaced, never thrown). */
|
|
225
|
+
readonly citations?: readonly {
|
|
226
|
+
readonly type: "url_citation";
|
|
227
|
+
readonly url: string;
|
|
228
|
+
readonly title?: string;
|
|
229
|
+
}[];
|
|
230
|
+
/** Native web_search tool calls billed on this grounded attempt (spec N3) —
|
|
231
|
+
* used by the raw success path to price the synthetic `rung:"search"` cost
|
|
232
|
+
* row. ADDITIVE: absent ⇒ the raw path assumes 1 (disclosed). */
|
|
233
|
+
readonly searchCalls?: number;
|
|
217
234
|
/** NATIVE attempt metadata (WI-1, spec §2) — finish reason, refusal,
|
|
218
235
|
* reasoning tokens, model snapshot, provider request id, sampling echo,
|
|
219
236
|
* tool-call names/counts. ADDITIVE: an adapter that reports nothing omits
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"provider.d.ts","sourceRoot":"","sources":["../../src/types/provider.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAC7C,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,2BAA2B,CAAC;AAE/D,MAAM,MAAM,UAAU,GAClB,QAAQ,GACR,QAAQ,GACR,UAAU,GACV,WAAW,GACX,MAAM,GAGN,WAAW,GACX,WAAW,CAAC;
|
|
1
|
+
{"version":3,"file":"provider.d.ts","sourceRoot":"","sources":["../../src/types/provider.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAC7C,OAAO,KAAK,EAAE,aAAa,EAAE,MAAM,2BAA2B,CAAC;AAE/D,MAAM,MAAM,UAAU,GAClB,QAAQ,GACR,QAAQ,GACR,UAAU,GACV,WAAW,GACX,MAAM,GAGN,WAAW,GACX,WAAW,GAIX,KAAK,CAAC;AAEV;;;;iBAIiB;AACjB,eAAO,MAAM,YAAY,iGASf,CAAC;AAYX,MAAM,MAAM,cAAc,GACtB,mBAAmB,GACnB,WAAW,GACX,iBAAiB,GACjB,wBAAwB,GACxB,kBAAkB,GAClB,uBAAuB,GACvB,WAAW,GACX,eAAe,GACf,gBAAgB,CAAC;AAErB;sEACsE;AACtE,eAAO,MAAM,oBAAoB,EAAE,SAAS,cAAc,EAUzD,CAAC;AAEF;;;;;;4EAM4E;AAC5E,MAAM,MAAM,mBAAmB,GAAG,aAAa,GAAG,WAAW,CAAC;AAM9D;oCACoC;AACpC,wBAAgB,iBAAiB,CAAC,QAAQ,EAAE,UAAU,GAAG,mBAAmB,CAE3E;AAsBD;oEACoE;AACpE,wBAAgB,qBAAqB,CAAC,QAAQ,EAAE,UAAU,GAAG,OAAO,CAEnE;AAED,OAAO,CAAC,MAAM,cAAc,EAAE,OAAO,MAAM,CAAC;AAE5C;;;;GAIG;AACH,MAAM,WAAW,YAAY;IAC3B,QAAQ,CAAC,QAAQ,EAAE,UAAU,CAAC;IAC9B,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,CAAC,cAAc,CAAC,EAAE,IAAI,CAAC;CACjC;AAED,MAAM,MAAM,WAAW,GAAG,UAAU,GAAG,MAAM,GAAG,UAAU,CAAC;AAE3D;;;;;;;;;;;GAWG;AACH,MAAM,WAAW,cAAc;IAC7B,2EAA2E;IAC3E,QAAQ,CAAC,WAAW,CAAC,EAAE,MAAM,CAAC;IAC9B,8EAA8E;IAC9E,QAAQ,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC;IACvB;8EAC0E;IAC1E,QAAQ,CAAC,eAAe,CAAC,EAAE,MAAM,CAAC;CACnC;AAED;;;;;;;;;2DAS2D;AAC3D,MAAM,MAAM,cAAc,GAAG,SAAS,GAAG,KAAK,GAAG,QAAQ,GAAG,MAAM,CAAC;AAEnE,MAAM,WAAW,eAAe;IAC9B,QAAQ,CAAC,KAAK,EAAE,cAAc,CAAC;CAChC;AASD;yDACyD;AACzD,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,IAAI,EAAE,UAAU,CAAC;IAC1B,QAAQ,CAAC,QAAQ,EAAE;QACjB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;QACtB,QAAQ,CAAC,WAAW,CAAC,EAAE,MAAM,CAAC;QAC9B,4CAA4C;QAC5C,QAAQ,CAAC,UAAU,CAAC,EAAE,MAAM,CAAC;KAC9B,CAAC;CACH;AAED;+EAC+E;AAC/E,MAAM,MAAM,UAAU,GAClB,MAAM,GACN,MAAM,GACN,UAAU,GACV;IAAE,QAAQ,CAAC,IAAI,EAAE,UAAU,CAAC;IAAC,QAAQ,CAAC,QAAQ,EAAE;QAAE,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;KAAE,CAAA;CAAE,CAAC;AAEhF;;;gBAGgB;AAChB,MAAM,WAAW,QAAQ;IACvB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,IAAI,EAAE,UAAU,CAAC;IAC1B,QAAQ,CAAC,QAAQ,EAAE;QACjB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;QACtB,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;KAC5B,CAAC;CACH;AAED,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,SAAS,EAAE,YAAY,CAAC;IACjC,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;IAC5B;;+EAE2E;IAC3E,QAAQ,CAAC,YAAY,EAAE,MAAM,GAAG,IAAI,CAAC;IACrC,QAAQ,CAAC,aAAa,EAAE,WAAW,CAAC;IACpC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,2EAA2E;IAC3E,QAAQ,CAAC,KAAK,CAAC,EAAE;QAAE,GAAG,EAAE,MAAM,CAAC;QAAC,UAAU,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IACtD;;;;;;OAMG;IACH,QAAQ,CAAC,SAAS,CAAC,EAAE,OAAO,CAAC;IAC7B,QAAQ,CAAC,eAAe,CAAC,EAAE,MAAM,CAAC;IAClC;;;;;;;;iCAQ6B;IAC7B,QAAQ,CAAC,QAAQ,CAAC,EAAE,cAAc,CAAC;IACnC;;;;;;;;;OASG;IACH,QAAQ,CAAC,SAAS,CAAC,EAAE,eAAe,CAAC;IACrC;;;;;;;;;;;8EAW0E;IAC1E,QAAQ,CAAC,cAAc,CAAC,EAAE,aAAa,CAAC;IACxC;gFAC4E;IAC5E,QAAQ,CAAC,KAAK,CAAC,EAAE,SAAS,cAAc,EAAE,CAAC;IAC3C,mEAAmE;IACnE,QAAQ,CAAC,UAAU,CAAC,EAAE,UAAU,CAAC;CAClC;AAED,MAAM,WAAW,YAAY;IAC3B;;;;;;;;OAQG;IACH,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,iFAAiF;IACjF,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B;;;;;;;OAOG;IACH,QAAQ,CAAC,gBAAgB,CAAC,EAAE,MAAM,CAAC;IACnC;;qEAEiE;IACjE,QAAQ,CAAC,kBAAkB,CAAC,EAAE,MAAM,CAAC;IACrC,QAAQ,CAAC,UAAU,EAAE,WAAW,CAAC;IACjC;;;;gCAI4B;IAC5B,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC;CAC1B;AAED,MAAM,MAAM,aAAa,GACrB;IACE,QAAQ,CAAC,EAAE,EAAE,IAAI,CAAC;IAClB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,KAAK,EAAE,YAAY,CAAC;IAC7B;;;uEAGmE;IACnE,QAAQ,CAAC,SAAS,CAAC,EAAE,SAAS,QAAQ,EAAE,CAAC;IACzC;;;;;+CAK2C;IAC3C,QAAQ,CAAC,SAAS,CAAC,EAAE,SAAS;QAAE,QAAQ,CAAC,IAAI,EAAE,cAAc,CAAC;QAAC,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,CAAA;KAAE,EAAE,CAAC;IACjH;;qEAEiE;IACjE,QAAQ,CAAC,WAAW,CAAC,EAAE,MAAM,CAAC;IAC9B;;;;;wCAKoC;IACpC,QAAQ,CAAC,IAAI,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,CAAC;CACxC;AACH;;;8EAG8E;GAC5E;IAAE,QAAQ,CAAC,EAAE,EAAE,KAAK,CAAC;IAAC,QAAQ,CAAC,OAAO,EAAE,UAAU,CAAC;IAAC,QAAQ,CAAC,KAAK,CAAC,EAAE,YAAY,CAAC;IAAC,QAAQ,CAAC,IAAI,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,CAAA;CAAE,CAAC;AAEhI;;;;4BAI4B;AAC5B,MAAM,MAAM,kBAAkB,GAC1B;IAAE,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;IAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GACjD;IAAE,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;IAAC,QAAQ,CAAC,MAAM,EAAE,aAAa,CAAA;CAAE,CAAC;AAE/D,MAAM,WAAW,eAAe;IAC9B,QAAQ,CAAC,EAAE,EAAE,UAAU,CAAC;IACxB,QAAQ,CAAC,OAAO,EAAE,cAAc,EAAE,KAAK,EAAE,MAAM,GAAG,OAAO,CAAC;IAC1D;;;qFAGiF;IACjF,aAAa,CAAC,CAAC,KAAK,EAAE,MAAM,GAAG,OAAO,CAAC;IACvC,IAAI,CAAC,GAAG,EAAE,cAAc,GAAG,OAAO,CAAC,aAAa,CAAC,CAAC;IAClD;mFAC+E;IAC/E,MAAM,CAAC,CAAC,GAAG,EAAE,cAAc,GAAG,aAAa,CAAC,kBAAkB,CAAC,CAAC;CACjE"}
|
package/dist/types/provider.js
CHANGED
package/dist/types/route.d.ts
CHANGED
|
@@ -3,11 +3,13 @@
|
|
|
3
3
|
* routes/loader.ts; frozen after load.
|
|
4
4
|
*/
|
|
5
5
|
import type { FailureClass } from "./failures";
|
|
6
|
+
import type { Destination } from "../run/destination";
|
|
6
7
|
import type { ProviderId, SamplingParams, ServiceTier, ToolCall, ToolChoice, ToolDefinition } from "./provider";
|
|
7
8
|
import type { QualityGate } from "./quality";
|
|
8
9
|
import type { MaskReceipt, ResolvedMaskPolicy } from "../mask/types";
|
|
9
10
|
import type { ResolvedScreenPolicy, ScreenReceipt } from "../screen/types";
|
|
10
11
|
import type { DataClassification } from "../artifacts/classification";
|
|
12
|
+
import type { Manifest } from "../prompts/manifest";
|
|
11
13
|
export interface CandidateSpec {
|
|
12
14
|
readonly provider: ProviderId;
|
|
13
15
|
readonly model: string;
|
|
@@ -191,16 +193,55 @@ export interface BakeoffConfig {
|
|
|
191
193
|
* when set true on a route with no reference. */
|
|
192
194
|
readonly upgradeWatch?: boolean;
|
|
193
195
|
}
|
|
196
|
+
/**
|
|
197
|
+
* Route OUTPUT KIND (GitLab issue #1, incident 2026-10-11). `json` (the default)
|
|
198
|
+
* is today's contract: the response MUST parse as JSON, then conform to
|
|
199
|
+
* `outputSchema` when one is declared. `text` is the prose/markdown contract:
|
|
200
|
+
* the response is returned VERBATIM as `output` (a string) with no JSON parse,
|
|
201
|
+
* so a route serving prose is never rejected `content_invalid` for not being
|
|
202
|
+
* JSON. Declared in the bundle YAML as `output: text` (see route-bundles.md).
|
|
203
|
+
*/
|
|
204
|
+
export type OutputKind = "json" | "text";
|
|
194
205
|
export interface RouteBundle {
|
|
195
206
|
readonly route: string;
|
|
196
207
|
readonly version: number;
|
|
197
208
|
/** Resolved, parsed JSON Schema (null = unstructured route). */
|
|
198
209
|
readonly outputSchema: object | null;
|
|
210
|
+
/** Output kind (issue #1). ONLY ever carried as `"text"` — the loader OMITS
|
|
211
|
+
* the field for a JSON route (absent ⇒ `json` ⇒ BYTE-IDENTICAL behaviour and
|
|
212
|
+
* an unchanged `bundle_sha` for every existing route). A `text` route has a
|
|
213
|
+
* null `outputSchema` (the loader refuses `output: text` + `schema:`), skips
|
|
214
|
+
* JSON parsing in `validateOutput`, and never enters the JSON repair rung.
|
|
215
|
+
* Carried on the resolved (mirror) bundle so the hosted lane honours it.
|
|
216
|
+
* Additive & optional. */
|
|
217
|
+
readonly outputKind?: OutputKind;
|
|
218
|
+
/** `output: { kind: text, min_chars: N }` — the one declared, fail-closed check
|
|
219
|
+
* a prose route may carry: a response shorter than N characters (untrimmed)
|
|
220
|
+
* is `content_invalid` (retried per the class budget, never JSON-repaired).
|
|
221
|
+
* Absent ⇒ no floor (any string, including empty, is accepted verbatim).
|
|
222
|
+
* Text routes only (load error otherwise). */
|
|
223
|
+
readonly outputMinChars?: number;
|
|
199
224
|
readonly candidates: readonly CandidateSpec[];
|
|
200
225
|
readonly require: readonly Requirement[];
|
|
201
226
|
readonly prefer: readonly Preference[];
|
|
202
227
|
/** Raw template text with {{var}} and {{#if capability.X}} blocks. */
|
|
203
228
|
readonly systemTemplate: string;
|
|
229
|
+
/**
|
|
230
|
+
* Per-call USER-turn template (GitLab issue #5, R1) — `prompt.user:` in the
|
|
231
|
+
* bundle YAML, rendered with the SAME `variables` / capability conditionals /
|
|
232
|
+
* `x-trust` removal as `systemTemplate`. When present, the rendered text IS the
|
|
233
|
+
* user turn of every primary attempt (replacing the fixed trigger sentence),
|
|
234
|
+
* and a same-model repair re-ask carries it ahead of the repair directive.
|
|
235
|
+
* This is how a route carries the per-call DELTA next to a customer cache
|
|
236
|
+
* handle: on a Gemini dispatch that sets `cachedContent` the adapter cannot
|
|
237
|
+
* send `systemInstruction` (the API rejects both), so the cache holds the
|
|
238
|
+
* shared prefix and `prompt.user` holds what changes per call — nothing is
|
|
239
|
+
* dropped. Present ONLY when declared (absent ⇒ the fixed trigger ⇒
|
|
240
|
+
* BYTE-IDENTICAL behaviour and an unchanged `bundle_sha`). Carried on the
|
|
241
|
+
* resolved (mirror) bundle so the hosted lane renders the same turn.
|
|
242
|
+
* Additive & optional.
|
|
243
|
+
*/
|
|
244
|
+
readonly userTemplate?: string;
|
|
204
245
|
readonly variables: readonly string[];
|
|
205
246
|
readonly policy: RoutePolicy;
|
|
206
247
|
/** Base-config repair rung (Phase 3 WS2) — variants may override. */
|
|
@@ -271,6 +312,24 @@ export interface RouteBundle {
|
|
|
271
312
|
readonly grade?: {
|
|
272
313
|
readonly passThreshold: number;
|
|
273
314
|
};
|
|
315
|
+
/** PSR-2 (spec §2.1/§3) — the served prompt's assembly manifest, structure-only.
|
|
316
|
+
* A promote writes the promoted `prompt_versions` row's `manifest` THROUGH into
|
|
317
|
+
* this bundle (+ recomputes `bundle_sha`) so `routes.bundle` is the served
|
|
318
|
+
* projection of the SoR. Carried on the wire so the SDK's client-fill engine
|
|
319
|
+
* (PSR-2b) can resolve each slot locally. Absent ⇒ no PSR manifest for this
|
|
320
|
+
* route (pre-PSR behavior, byte-identical). Structure only — no resolved values
|
|
321
|
+
* (D6). Additive & optional. */
|
|
322
|
+
readonly manifest?: Manifest;
|
|
323
|
+
/** Unified Route U2 (ruling D-2) — the serving `destination` for this route:
|
|
324
|
+
* `direct` (today's behaviour: the configured/BYOK key) or `modelrig-served`
|
|
325
|
+
* (ModelRig serves the model on its OWN managed key). Validated at load
|
|
326
|
+
* (routes/loader.ts) against the DESTINATIONS vocabulary. Absent ⇒ resolves to
|
|
327
|
+
* `direct` ⇒ BYTE-IDENTICAL (the top acceptance bar). A `RunOptions.destination`
|
|
328
|
+
* per-run override wins over this. In U2 a resolved `modelrig-served` FAILS
|
|
329
|
+
* CLOSED (managed serving is gated behind the `config.managedServing` flag, U3)
|
|
330
|
+
* rather than silently serving `direct` — fail-closed-but-discoverable. Additive
|
|
331
|
+
* & optional. */
|
|
332
|
+
readonly destination?: Destination;
|
|
274
333
|
}
|
|
275
334
|
export interface RunTags {
|
|
276
335
|
readonly run_id?: string;
|
|
@@ -386,6 +445,13 @@ export interface RunOptions {
|
|
|
386
445
|
* rule ships as validation + tests; the structural flip is B3b).
|
|
387
446
|
*/
|
|
388
447
|
readonly proxyTrust?: readonly string[];
|
|
448
|
+
/** Unified Route U2 (ruling D-2) — per-run serving `destination` override. Wins
|
|
449
|
+
* over the route's declared `destination` when resolving the effective served
|
|
450
|
+
* destination (RunOptions > route > default `direct`). Absent ⇒ the route's
|
|
451
|
+
* declared `destination` stands (itself absent ⇒ `direct` ⇒ BYTE-IDENTICAL).
|
|
452
|
+
* The hosted gateway's YAML→RunOptions mapping of this key is a
|
|
453
|
+
* `modelrig-server` follow-up, out of this package's scope. Additive & optional. */
|
|
454
|
+
readonly destination?: Destination;
|
|
389
455
|
}
|
|
390
456
|
export interface RunMeta {
|
|
391
457
|
/** Telemetry row id of the serving attempt — the id a grade attaches to via
|
|
@@ -453,6 +519,16 @@ export interface RunMeta {
|
|
|
453
519
|
* matched value. Rides `inferences.meta.screen` in telemetry (spec §5, §8 — no
|
|
454
520
|
* new column). Additive & optional. */
|
|
455
521
|
readonly screen?: ScreenReceipt;
|
|
522
|
+
/** Unified Route U2 (ruling D-2) — the RESOLVED served destination for this run
|
|
523
|
+
* (RunOptions override > route > default `direct`). Additive RESULT metadata,
|
|
524
|
+
* like `servedTier`: the routed lane populates it on every returned run so
|
|
525
|
+
* telemetry/console can read WHERE the call was served. This is NOT the wire
|
|
526
|
+
* telemetry row — stamping `destination` on the row is gated to a NON-default
|
|
527
|
+
* value (destinationTag), so an absent/`direct` route's row is byte-identical.
|
|
528
|
+
* The raw inline on-ramp omits it (its destination is always `direct`
|
|
529
|
+
* implicitly). In U2 a returned run's value is always `direct` (a resolved
|
|
530
|
+
* `modelrig-served` fails closed pre-dispatch). Additive & optional. */
|
|
531
|
+
readonly destination?: Destination;
|
|
456
532
|
}
|
|
457
533
|
export interface RunResult {
|
|
458
534
|
readonly output: unknown;
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"route.d.ts","sourceRoot":"","sources":["../../src/types/route.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAC/C,OAAO,KAAK,EAAE,UAAU,EAAE,cAAc,EAAE,WAAW,EAAE,QAAQ,EAAE,UAAU,EAAE,cAAc,EAAE,MAAM,YAAY,CAAC;AAChH,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,WAAW,CAAC;AAC7C,OAAO,KAAK,EAAE,WAAW,EAAE,kBAAkB,EAAE,MAAM,eAAe,CAAC;AACrE,OAAO,KAAK,EAAE,oBAAoB,EAAE,aAAa,EAAE,MAAM,iBAAiB,CAAC;AAC3E,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,6BAA6B,CAAC;
|
|
1
|
+
{"version":3,"file":"route.d.ts","sourceRoot":"","sources":["../../src/types/route.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAC/C,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,oBAAoB,CAAC;AACtD,OAAO,KAAK,EAAE,UAAU,EAAE,cAAc,EAAE,WAAW,EAAE,QAAQ,EAAE,UAAU,EAAE,cAAc,EAAE,MAAM,YAAY,CAAC;AAChH,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,WAAW,CAAC;AAC7C,OAAO,KAAK,EAAE,WAAW,EAAE,kBAAkB,EAAE,MAAM,eAAe,CAAC;AACrE,OAAO,KAAK,EAAE,oBAAoB,EAAE,aAAa,EAAE,MAAM,iBAAiB,CAAC;AAC3E,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,6BAA6B,CAAC;AACtE,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,qBAAqB,CAAC;AAEpD,MAAM,WAAW,aAAa;IAC5B,QAAQ,CAAC,QAAQ,EAAE,UAAU,CAAC;IAC9B,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB;;;;;;;;;;;;;;;OAeG;IACH,QAAQ,CAAC,KAAK,CAAC,EAAE,OAAO,CAAC;CAC1B;AAED,MAAM,MAAM,WAAW,GAAG,mBAAmB,GAAG,UAAU,GAAG,gBAAgB,GAAG,eAAe,CAAC;AAChG,MAAM,MAAM,UAAU,GAAG,MAAM,GAAG,SAAS,CAAC;AAE5C,MAAM,WAAW,WAAW;IAC1B,0EAA0E;IAC1E,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAC,MAAM,CAAC,YAAY,EAAE,MAAM,CAAC,CAAC,CAAC;IACxD,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,IAAI,EAAE,WAAW,CAAC;IAC3B,QAAQ,CAAC,IAAI,EAAE,QAAQ,GAAG,WAAW,CAAC;IACtC;;;;;;;mCAO+B;IAC/B,QAAQ,CAAC,QAAQ,CAAC,EAAE,cAAc,CAAC;CACpC;AAED,uDAAuD;AACvD,MAAM,WAAW,YAAY;IAC3B;;;kCAG8B;IAC9B,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;IAC5B;oEACgE;IAChE,QAAQ,CAAC,WAAW,CAAC,EAAE,aAAa,CAAC;CACtC;AAED;6EAC6E;AAC7E,MAAM,WAAW,YAAY;IAC3B,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB;;wBAEoB;IACpB,QAAQ,CAAC,UAAU,CAAC,EAAE,SAAS,aAAa,EAAE,CAAC;IAC/C;6EACyE;IACzE,QAAQ,CAAC,cAAc,CAAC,EAAE,MAAM,CAAC;IACjC;gCAC4B;IAC5B,QAAQ,CAAC,YAAY,CAAC,EAAE,MAAM,CAAC;IAC/B;4DACwD;IACxD,QAAQ,CAAC,QAAQ,CAAC,EAAE,MAAM,CAAC;IAC3B,2CAA2C;IAC3C,QAAQ,CAAC,IAAI,CAAC,EAAE,QAAQ,GAAG,WAAW,CAAC;IACvC,QAAQ,CAAC,MAAM,CAAC,EAAE,YAAY,CAAC;IAC/B;;;;;;OAMG;IACH,QAAQ,CAAC,SAAS,CAAC,EAAE,OAAO,CAAC;CAC9B;AAED;;;;+CAI+C;AAC/C,MAAM,MAAM,aAAa,GAAG,eAAe,GAAG,UAAU,GAAG,YAAY,CAAC;AAExE;;;8BAG8B;AAC9B,MAAM,WAAW,YAAY;IAC3B,QAAQ,CAAC,QAAQ,CAAC,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAC;CAC7B;AAED,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,IAAI,EAAE,aAAa,CAAC;IAC7B,gFAAgF;IAChF,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAC;IAC5B,QAAQ,CAAC,YAAY,CAAC,EAAE,YAAY,CAAC;IACrC,kFAAkF;IAClF,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC;IACzB;;;;;;;OAOG;IACH,QAAQ,CAAC,aAAa,CAAC,EAAE,OAAO,CAAC;CAClC;AAED;;;+EAG+E;AAC/E,MAAM,WAAW,UAAU;IACzB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,MAAM,EAAE;QAAE,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAA;KAAE,GAAG;QAAE,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;KAAE,CAAC;CAC7E;AAED;0DAC0D;AAC1D,MAAM,WAAW,aAAa;IAC5B;;wFAEoF;IACpF,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,GAAG;QAAE,QAAQ,CAAC,QAAQ,EAAE,UAAU,CAAC;QAAC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;KAAE,CAAC;IACxF,2EAA2E;IAC3E,QAAQ,CAAC,sBAAsB,CAAC,EAAE,OAAO,CAAC;IAC1C,qEAAqE;IACrE,QAAQ,CAAC,oBAAoB,CAAC,EAAE,MAAM,CAAC;IACvC;;;+BAG2B;IAC3B,QAAQ,CAAC,KAAK,CAAC,EAAE;QAAE,QAAQ,CAAC,KAAK,EAAE,aAAa,CAAC;QAAC,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;KAAE,CAAC;IAChF;;;;;;;;;OASG;IACH,QAAQ,CAAC,QAAQ,CAAC,EAAE;QAAE,QAAQ,CAAC,KAAK,CAAC,EAAE;YAAE,QAAQ,CAAC,KAAK,EAAE,aAAa,CAAC;YAAC,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;SAAE,CAAA;KAAE,CAAC;IACxG;;;;;;yEAMqE;IACrE,QAAQ,CAAC,aAAa,CAAC,EAAE;QACvB,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAC;QAC1B,qEAAqE;QACrE,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAC;QAC1B,QAAQ,CAAC,UAAU,CAAC,EAAE,MAAM,CAAC;QAC7B,QAAQ,CAAC,YAAY,CAAC,EAAE,MAAM,CAAC;KAChC,CAAC;IACF;;;;;;qDAMiD;IACjD,QAAQ,CAAC,YAAY,CAAC,EAAE,OAAO,CAAC;CACjC;AAED;;;;;;;GAOG;AACH,MAAM,MAAM,UAAU,GAAG,MAAM,GAAG,MAAM,CAAC;AAEzC,MAAM,WAAW,WAAW;IAC1B,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,gEAAgE;IAChE,QAAQ,CAAC,YAAY,EAAE,MAAM,GAAG,IAAI,CAAC;IACrC;;;;;;8BAM0B;IAC1B,QAAQ,CAAC,UAAU,CAAC,EAAE,UAAU,CAAC;IACjC;;;;kDAI8C;IAC9C,QAAQ,CAAC,cAAc,CAAC,EAAE,MAAM,CAAC;IACjC,QAAQ,CAAC,UAAU,EAAE,SAAS,aAAa,EAAE,CAAC;IAC9C,QAAQ,CAAC,OAAO,EAAE,SAAS,WAAW,EAAE,CAAC;IACzC,QAAQ,CAAC,MAAM,EAAE,SAAS,UAAU,EAAE,CAAC;IACvC,sEAAsE;IACtE,QAAQ,CAAC,cAAc,EAAE,MAAM,CAAC;IAChC;;;;;;;;;;;;;;OAcG;IACH,QAAQ,CAAC,YAAY,CAAC,EAAE,MAAM,CAAC;IAC/B,QAAQ,CAAC,SAAS,EAAE,SAAS,MAAM,EAAE,CAAC;IACtC,QAAQ,CAAC,MAAM,EAAE,WAAW,CAAC;IAC7B,qEAAqE;IACrE,QAAQ,CAAC,MAAM,CAAC,EAAE,YAAY,CAAC;IAC/B,mEAAmE;IACnE,QAAQ,CAAC,QAAQ,EAAE,SAAS,YAAY,EAAE,CAAC;IAC3C,sEAAsE;IACtE,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAC;IAC1B;;;;;;uCAMmC;IACnC,QAAQ,CAAC,KAAK,CAAC,EAAE,OAAO,CAAC;IACzB;;wEAEoE;IACpE,QAAQ,CAAC,cAAc,CAAC,EAAE,MAAM,CAAC;IACjC;kEAC8D;IAC9D,QAAQ,CAAC,eAAe,EAAE,MAAM,GAAG,IAAI,CAAC;IACxC;4BACwB;IACxB,QAAQ,CAAC,SAAS,CAAC,EAAE,cAAc,CAAC;IACpC,sDAAsD;IACtD,QAAQ,CAAC,OAAO,CAAC,EAAE,aAAa,CAAC;IACjC,kEAAkE;IAClE,QAAQ,CAAC,WAAW,CAAC,EAAE,SAAS,UAAU,EAAE,CAAC;IAC7C;;;;;0EAKsE;IACtE,QAAQ,CAAC,iBAAiB,CAAC,EAAE,QAAQ,GAAG,WAAW,CAAC;IACpD;;;iFAG6E;IAC7E,QAAQ,CAAC,aAAa,CAAC,EAAE,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,WAAW,CAAC,CAAC,CAAC;IAC/D;;;;sFAIkF;IAClF,QAAQ,CAAC,IAAI,CAAC,EAAE,kBAAkB,CAAC;IACnC;;;;;;wCAMoC;IACpC,QAAQ,CAAC,MAAM,CAAC,EAAE,oBAAoB,CAAC;IACvC;;;;6EAIyE;IACzE,QAAQ,CAAC,cAAc,CAAC,EAAE,kBAAkB,CAAC;IAC7C;;;;8BAI0B;IAC1B,QAAQ,CAAC,KAAK,CAAC,EAAE;QAAE,QAAQ,CAAC,aAAa,EAAE,MAAM,CAAA;KAAE,CAAC;IACpD;;;;;;oCAMgC;IAChC,QAAQ,CAAC,QAAQ,CAAC,EAAE,QAAQ,CAAC;IAC7B;;;;;;;;qBAQiB;IACjB,QAAQ,CAAC,WAAW,CAAC,EAAE,WAAW,CAAC;CACpC;AAED,MAAM,WAAW,OAAO;IACtB,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC;IACzB,QAAQ,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC;IACvB,QAAQ,EAAE,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAAC;CAC5C;AAED,MAAM,WAAW,UAAU;IACzB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACxC,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;IACvB;;;;mFAI+E;IAC/E,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,MAAM,CAAC,EAAE;QAChB,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;QAC1B;;;;kFAI0E;QAC1E,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAC;KAC7B,CAAC;IACF;qEACiE;IACjE,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAC;IAC1B;;;;oFAIgF;IAChF,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,CAAC;IACtB;;;;;;oCAMgC;IAChC,QAAQ,CAAC,aAAa,CAAC,EAAE,OAAO,CAAC;IACjC;;;;wEAIoE;IACpE,QAAQ,CAAC,kBAAkB,CAAC,EAAE,MAAM,CAAC;IACrC,0EAA0E;IAC1E,QAAQ,CAAC,SAAS,CAAC,EAAE;QACnB,QAAQ,CAAC,IAAI,CAAC,EAAE,SAAS,UAAU,EAAE,CAAC;QACtC,QAAQ,CAAC,MAAM,CAAC,EAAE,SAAS,UAAU,EAAE,CAAC;KACzC,CAAC;IACF;;;uDAGmD;IACnD,QAAQ,CAAC,KAAK,CAAC,EAAE,SAAS,cAAc,EAAE,CAAC;IAC3C,QAAQ,CAAC,UAAU,CAAC,EAAE,UAAU,CAAC;IACjC;;;;;;;;;;OAUG;IACH,QAAQ,CAAC,KAAK,CAAC,EAAE;QACf,iEAAiE;QACjE,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAC;QACrB;;;;;;iCAMyB;QACzB,QAAQ,CAAC,QAAQ,EAAE,UAAU,CAAC;QAC9B;2EACmE;QACnE,QAAQ,CAAC,UAAU,CAAC,EAAE,MAAM,CAAC;KAC9B,CAAC;IACF;;;;;;;;;;;;OAYG;IACH,QAAQ,CAAC,WAAW,CAAC,EAAE,WAAW,CAAC;IACnC;;;;;;;;;;;;;;;OAeG;IACH,QAAQ,CAAC,UAAU,CAAC,EAAE,SAAS,MAAM,EAAE,CAAC;IACxC;;;;;wFAKoF;IACpF,QAAQ,CAAC,WAAW,CAAC,EAAE,WAAW,CAAC;CACpC;AAED,MAAM,WAAW,OAAO;IACtB;iDAC6C;IAC7C,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,QAAQ,EAAE,UAAU,CAAC;IAC9B,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,aAAa,EAAE,WAAW,CAAC;IACpC,QAAQ,CAAC,UAAU,EAAE,WAAW,CAAC;IACjC,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAC;IACjC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,SAAS,EAAE,OAAO,CAAC;IAC5B,QAAQ,CAAC,eAAe,EAAE,OAAO,CAAC,MAAM,CAAC,YAAY,EAAE,MAAM,CAAC,CAAC,CAAC;IAChE,2EAA2E;IAC3E,QAAQ,CAAC,aAAa,EAAE,MAAM,GAAG,IAAI,CAAC;IACtC;wBACoB;IACpB,QAAQ,CAAC,UAAU,EAAE,MAAM,GAAG,IAAI,CAAC;IACnC;;;+EAG2E;IAC3E,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC;IACzB;gDAC4C;IAC5C,QAAQ,CAAC,SAAS,CAAC,EAAE,aAAa,CAAC;QACjC,QAAQ,CAAC,IAAI,EAAE,cAAc,CAAC;QAC9B,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAC;QACrB,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC;KACzB,CAAC,CAAC;IACH;;iFAE6E;IAC7E,QAAQ,CAAC,SAAS,CAAC,EAAE;QAAE,QAAQ,CAAC,iBAAiB,EAAE,MAAM,CAAA;KAAE,CAAC;IAC5D;;gFAE4E;IAC5E,QAAQ,CAAC,SAAS,CAAC,EAAE,SAAS,QAAQ,EAAE,CAAC;IACzC;;;;iFAI6E;IAC7E,QAAQ,CAAC,OAAO,CAAC,EAAE;QACjB,QAAQ,CAAC,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;QAC9B,QAAQ,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,CAAC;QAClC,QAAQ,CAAC,QAAQ,EAAE,MAAM,GAAG,IAAI,CAAC;KAClC,CAAC;IACF;;;6EAGyE;IACzE,QAAQ,CAAC,IAAI,CAAC,EAAE,WAAW,CAAC;IAC5B;;;;;2CAKuC;IACvC,QAAQ,CAAC,MAAM,CAAC,EAAE,aAAa,CAAC;IAChC;;;;;;;;4EAQwE;IACxE,QAAQ,CAAC,WAAW,CAAC,EAAE,WAAW,CAAC;CACpC;AAED,MAAM,WAAW,SAAS;IACxB,QAAQ,CAAC,MAAM,EAAE,OAAO,CAAC;IACzB,QAAQ,CAAC,IAAI,EAAE,OAAO,CAAC;CACxB"}
|
|
@@ -113,6 +113,25 @@ export interface EnvelopeRow {
|
|
|
113
113
|
readonly spentUsd: number;
|
|
114
114
|
readonly status: "open" | "exhausted";
|
|
115
115
|
}
|
|
116
|
+
/** One evidence citation on a per-criterion grade — a verbatim quote drawn from
|
|
117
|
+
* the subject, optionally anchored to a transcript turn (call-qa F2/F4). */
|
|
118
|
+
export interface GradeCriterionEvidence {
|
|
119
|
+
readonly quote: string;
|
|
120
|
+
readonly turn_id?: number;
|
|
121
|
+
}
|
|
122
|
+
/** One criterion's outcome on a HUMAN scorecard (call-qa F4 — the human twin of
|
|
123
|
+
* the server's model-emitted ScorecardCriterion). `score` is the normalized 0–1
|
|
124
|
+
* sub-score; `evidence`/`comment` are the analyst's grounded justification. A
|
|
125
|
+
* human grade may OPTIONALLY carry an array of these; when present the server
|
|
126
|
+
* persists them to `grade_criteria` (kind semantics = human) so per-criterion
|
|
127
|
+
* agreement (F4 calibration) has a human side to pair. Absent ⇒ today's
|
|
128
|
+
* single-score human grade, unchanged. */
|
|
129
|
+
export interface GradeCriterion {
|
|
130
|
+
readonly id: string;
|
|
131
|
+
readonly score: number;
|
|
132
|
+
readonly evidence?: readonly GradeCriterionEvidence[];
|
|
133
|
+
readonly comment?: string;
|
|
134
|
+
}
|
|
116
135
|
/** The subject a grade attaches to (grade primitive) — an inference, a run
|
|
117
136
|
* (episode), or an artifact. Exactly one, by construction. */
|
|
118
137
|
export type GradeSubject = {
|
|
@@ -156,6 +175,13 @@ export interface GradeRow {
|
|
|
156
175
|
readonly createdBy: string | null;
|
|
157
176
|
/** Judgment timestamp (feedback carried one; evaluations fall back to created_at). */
|
|
158
177
|
readonly ts: string | null;
|
|
178
|
+
/** OPTIONAL per-criterion scorecard (call-qa F4). Present only on a human grade
|
|
179
|
+
* carrying a breakdown; the exporter mirrors it on the wire (WireGrade.criteria)
|
|
180
|
+
* and the server persists it to `grade_criteria` (kind=human) after the parent
|
|
181
|
+
* grades row lands. Absent ⇒ today's single-score grade — no grade_criteria.
|
|
182
|
+
* Additive: stored locally as a JSON `criteria` TEXT column (v16), read back as
|
|
183
|
+
* null for any row written before F4. */
|
|
184
|
+
readonly criteria?: readonly GradeCriterion[];
|
|
159
185
|
}
|
|
160
186
|
/** One execution of a customer pipeline (Expansion E2 chunk A). Local SQLite is
|
|
161
187
|
* the write path (run.start/run.end); the exporter mirrors it to `public.runs`
|
|
@@ -226,8 +252,10 @@ export interface ArtifactEdgeRow {
|
|
|
226
252
|
readonly toArtifactId: string;
|
|
227
253
|
readonly kind: "consumed" | "derived_from";
|
|
228
254
|
}
|
|
229
|
-
/** How a case entered the eval suite (E4 chunk A, migration 00040 CHECK
|
|
230
|
-
|
|
255
|
+
/** How a case entered the eval suite (E4 chunk A, migration 00040 CHECK; the
|
|
256
|
+
* imported-* origins added by the dataset-source ingest thread, migration 00078
|
|
257
|
+
* CHECK). */
|
|
258
|
+
export type EvalCaseOrigin = "promoted-failure" | "human-corrected" | "manual" | "imported-ticket" | "imported-chat";
|
|
231
259
|
/** A per-org regression/golden eval case (E4 chunk A). `correctedArtifactId`
|
|
232
260
|
* null ⇒ a failure-case; set ⇒ a golden-corrected pair. A fact on ingest
|
|
233
261
|
* (created via promote); edits/deletes are server-side, not on this stream. */
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"telemetry.d.ts","sourceRoot":"","sources":["../../src/types/telemetry.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAC/C,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,SAAS,CAAC;AACvC,OAAO,KAAK,EAAE,aAAa,EAAE,SAAS,EAAE,MAAM,2BAA2B,CAAC;AAC1E,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AAEjD;;;;;;;;;;;;;GAaG;AACH,MAAM,MAAM,aAAa,GACrB,SAAS,GACT,OAAO,GACP,QAAQ,GACR,QAAQ,GACR,WAAW,GACX,kBAAkB,CAAC;AAEvB,MAAM,WAAW,YAAY;IAC3B,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,aAAa,EAAE,MAAM,CAAC;IAC/B,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;IAC5B;;wCAEoC;IACpC,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,sDAAsD;IACtD,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B;;;;;;kFAM8E;IAC9E,QAAQ,CAAC,gBAAgB,CAAC,EAAE,MAAM,CAAC;IACnC;wFACoF;IACpF,QAAQ,CAAC,kBAAkB,CAAC,EAAE,MAAM,CAAC;IACrC,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAC;IACjC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B;uDACmD;IACnD,QAAQ,CAAC,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;IAC/B,QAAQ,CAAC,YAAY,EAAE,YAAY,GAAG,IAAI,CAAC;IAC3C,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B;wEACoE;IACpE,QAAQ,CAAC,eAAe,EAAE,MAAM,GAAG,IAAI,CAAC;IACxC;;;oDAGgD;IAChD,QAAQ,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAClC;oFACgF;IAChF,QAAQ,CAAC,iBAAiB,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC3C;;6DAEyD;IACzD,QAAQ,CAAC,cAAc,EAAE,OAAO,CAAC;IACjC;oDACgD;IAChD,QAAQ,CAAC,aAAa,EAAE,MAAM,GAAG,IAAI,CAAC;IACtC;uDACmD;IACnD,QAAQ,CAAC,UAAU,EAAE,MAAM,GAAG,IAAI,CAAC;IACnC;;;;;;+EAM2E;IAC3E,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAChC;mEAC+D;IAC/D,QAAQ,CAAC,UAAU,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACpC;;;sDAGkD;IAClD,QAAQ,CAAC,cAAc,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACxC;;;;;uDAKmD;IACnD,QAAQ,CAAC,IAAI,CAAC,EAAE,aAAa,GAAG,IAAI,CAAC;IACrC;;;;uEAImE;IACnE,QAAQ,CAAC,IAAI,CAAC,EAAE,WAAW,GAAG,IAAI,CAAC;CACpC;AAED,MAAM,WAAW,WAAW;IAC1B,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,MAAM,EAAE,MAAM,GAAG,WAAW,CAAC;CACvC;AAED;8DAC8D;AAC9D,MAAM,MAAM,YAAY,GACpB;IAAE,QAAQ,CAAC,IAAI,EAAE,WAAW,CAAC;IAAC,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAA;CAAE,GACnD;IAAE,QAAQ,CAAC,IAAI,EAAE,KAAK,CAAC;IAAC,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAA;CAAE,GAC7C;IAAE,QAAQ,CAAC,IAAI,EAAE,UAAU,CAAC;IAAC,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAA;CAAE,CAAC;AAEvD;;;;4EAI4E;AAC5E,MAAM,WAAW,QAAQ;IACvB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,WAAW,EAAE,WAAW,GAAG,KAAK,GAAG,UAAU,CAAC;IACvD,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B;yFACqF;IACrF,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,8EAA8E;IAC9E,QAAQ,CAAC,QAAQ,EAAE,MAAM,GAAG,IAAI,CAAC;IACjC;;oEAEgE;IAChE,QAAQ,CAAC,UAAU,EAAE,MAAM,GAAG,IAAI,CAAC;IACnC,qEAAqE;IACrE,QAAQ,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IAChC,QAAQ,CAAC,IAAI,EAAE,OAAO,GAAG,aAAa,GAAG,eAAe,CAAC;IACzD,iFAAiF;IACjF,QAAQ,CAAC,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;IAC/B,gFAAgF;IAChF,QAAQ,CAAC,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;IAC/B;mFAC+E;IAC/E,QAAQ,CAAC,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;IAC9B,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IAClC,sFAAsF;IACtF,QAAQ,CAAC,EAAE,EAAE,MAAM,GAAG,IAAI,CAAC;
|
|
1
|
+
{"version":3,"file":"telemetry.d.ts","sourceRoot":"","sources":["../../src/types/telemetry.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAC/C,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,SAAS,CAAC;AACvC,OAAO,KAAK,EAAE,aAAa,EAAE,SAAS,EAAE,MAAM,2BAA2B,CAAC;AAC1E,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AAEjD;;;;;;;;;;;;;GAaG;AACH,MAAM,MAAM,aAAa,GACrB,SAAS,GACT,OAAO,GACP,QAAQ,GACR,QAAQ,GACR,WAAW,GACX,kBAAkB,CAAC;AAEvB,MAAM,WAAW,YAAY;IAC3B,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,aAAa,EAAE,MAAM,CAAC;IAC/B,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;IAC5B;;wCAEoC;IACpC,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,sDAAsD;IACtD,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B;;;;;;kFAM8E;IAC9E,QAAQ,CAAC,gBAAgB,CAAC,EAAE,MAAM,CAAC;IACnC;wFACoF;IACpF,QAAQ,CAAC,kBAAkB,CAAC,EAAE,MAAM,CAAC;IACrC,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAC;IACjC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B;uDACmD;IACnD,QAAQ,CAAC,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;IAC/B,QAAQ,CAAC,YAAY,EAAE,YAAY,GAAG,IAAI,CAAC;IAC3C,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B;wEACoE;IACpE,QAAQ,CAAC,eAAe,EAAE,MAAM,GAAG,IAAI,CAAC;IACxC;;;oDAGgD;IAChD,QAAQ,CAAC,QAAQ,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAClC;oFACgF;IAChF,QAAQ,CAAC,iBAAiB,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC3C;;6DAEyD;IACzD,QAAQ,CAAC,cAAc,EAAE,OAAO,CAAC;IACjC;oDACgD;IAChD,QAAQ,CAAC,aAAa,EAAE,MAAM,GAAG,IAAI,CAAC;IACtC;uDACmD;IACnD,QAAQ,CAAC,UAAU,EAAE,MAAM,GAAG,IAAI,CAAC;IACnC;;;;;;+EAM2E;IAC3E,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAChC;mEAC+D;IAC/D,QAAQ,CAAC,UAAU,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACpC;;;sDAGkD;IAClD,QAAQ,CAAC,cAAc,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACxC;;;;;uDAKmD;IACnD,QAAQ,CAAC,IAAI,CAAC,EAAE,aAAa,GAAG,IAAI,CAAC;IACrC;;;;uEAImE;IACnE,QAAQ,CAAC,IAAI,CAAC,EAAE,WAAW,GAAG,IAAI,CAAC;CACpC;AAED,MAAM,WAAW,WAAW;IAC1B,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,MAAM,EAAE,MAAM,GAAG,WAAW,CAAC;CACvC;AAED;4EAC4E;AAC5E,MAAM,WAAW,sBAAsB;IACrC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAC;CAC3B;AAED;;;;;;0CAM0C;AAC1C,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,QAAQ,CAAC,EAAE,SAAS,sBAAsB,EAAE,CAAC;IACtD,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAC;CAC3B;AAED;8DAC8D;AAC9D,MAAM,MAAM,YAAY,GACpB;IAAE,QAAQ,CAAC,IAAI,EAAE,WAAW,CAAC;IAAC,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAA;CAAE,GACnD;IAAE,QAAQ,CAAC,IAAI,EAAE,KAAK,CAAC;IAAC,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAA;CAAE,GAC7C;IAAE,QAAQ,CAAC,IAAI,EAAE,UAAU,CAAC;IAAC,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAA;CAAE,CAAC;AAEvD;;;;4EAI4E;AAC5E,MAAM,WAAW,QAAQ;IACvB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,WAAW,EAAE,WAAW,GAAG,KAAK,GAAG,UAAU,CAAC;IACvD,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B;yFACqF;IACrF,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,8EAA8E;IAC9E,QAAQ,CAAC,QAAQ,EAAE,MAAM,GAAG,IAAI,CAAC;IACjC;;oEAEgE;IAChE,QAAQ,CAAC,UAAU,EAAE,MAAM,GAAG,IAAI,CAAC;IACnC,qEAAqE;IACrE,QAAQ,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IAChC,QAAQ,CAAC,IAAI,EAAE,OAAO,GAAG,aAAa,GAAG,eAAe,CAAC;IACzD,iFAAiF;IACjF,QAAQ,CAAC,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;IAC/B,gFAAgF;IAChF,QAAQ,CAAC,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;IAC/B;mFAC+E;IAC/E,QAAQ,CAAC,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;IAC9B,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IAClC,sFAAsF;IACtF,QAAQ,CAAC,EAAE,EAAE,MAAM,GAAG,IAAI,CAAC;IAC3B;;;;;6CAKyC;IACzC,QAAQ,CAAC,QAAQ,CAAC,EAAE,SAAS,cAAc,EAAE,CAAC;CAC/C;AAED;;;;cAIc;AACd,MAAM,WAAW,MAAM;IACrB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,eAAe,EAAE,MAAM,GAAG,IAAI,CAAC;IACxC,QAAQ,CAAC,WAAW,EAAE,MAAM,GAAG,IAAI,CAAC;IACpC,QAAQ,CAAC,UAAU,EAAE,MAAM,GAAG,IAAI,CAAC;IACnC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IAChC,QAAQ,CAAC,MAAM,EAAE,SAAS,GAAG,WAAW,GAAG,QAAQ,GAAG,WAAW,CAAC;IAClE;;;6EAGyE;IACzE,QAAQ,CAAC,IAAI,CAAC,EAAE,SAAS,GAAG,IAAI,CAAC;CAClC;AAED;;;gDAGgD;AAChD,MAAM,WAAW,OAAO;IACtB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAC;IAC7B,QAAQ,CAAC,UAAU,EAAE,cAAc,GAAG,UAAU,CAAC;IACjD,QAAQ,CAAC,WAAW,EAAE,MAAM,GAAG,IAAI,CAAC;IACpC,QAAQ,CAAC,WAAW,EAAE,MAAM,GAAG,IAAI,CAAC;IACpC,QAAQ,CAAC,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;IAC/B,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IAChC,QAAQ,CAAC,YAAY,EAAE,MAAM,GAAG,IAAI,CAAC;CACtC;AAED;;mFAEmF;AACnF,MAAM,WAAW,WAAW;IAC1B,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,KAAK,EAAE,MAAM,GAAG,IAAI,CAAC;IAC9B,QAAQ,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,CAAC;IAChC,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IAClC,QAAQ,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAC;IAC7B,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,QAAQ,CAAC,oBAAoB,EAAE,MAAM,GAAG,IAAI,CAAC;IAC7C,QAAQ,CAAC,cAAc,EAAE,OAAO,CAAC;IACjC,QAAQ,CAAC,eAAe,EAAE,MAAM,GAAG,IAAI,CAAC;IACxC,QAAQ,CAAC,UAAU,EAAE,MAAM,GAAG,IAAI,CAAC;IACnC,QAAQ,CAAC,WAAW,EAAE,MAAM,GAAG,IAAI,CAAC;IACpC,QAAQ,CAAC,YAAY,EAAE,MAAM,GAAG,IAAI,CAAC;IACrC,QAAQ,CAAC,kBAAkB,EAAE,MAAM,GAAG,IAAI,CAAC;IAC3C,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IAClC;;;kEAG8D;IAC9D,QAAQ,CAAC,KAAK,CAAC,EAAE,OAAO,CAAC;CAC1B;AAED,sDAAsD;AACtD,MAAM,WAAW,eAAe;IAC9B,QAAQ,CAAC,cAAc,EAAE,MAAM,CAAC;IAChC,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAC;IAC9B,QAAQ,CAAC,IAAI,EAAE,UAAU,GAAG,cAAc,CAAC;CAC5C;AAED;;aAEa;AACb,MAAM,MAAM,cAAc,GACtB,kBAAkB,GAClB,iBAAiB,GACjB,QAAQ,GACR,iBAAiB,GACjB,eAAe,CAAC;AAEpB;;+EAE+E;AAC/E,MAAM,WAAW,WAAW;IAC1B,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,gBAAgB,EAAE,MAAM,CAAC;IAClC,QAAQ,CAAC,mBAAmB,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5C,QAAQ,CAAC,MAAM,EAAE,cAAc,CAAC;IAChC,QAAQ,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAC;IAC7B,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;CACnC;AAED,MAAM,WAAW,cAAc;IAC7B,MAAM,CAAC,GAAG,EAAE,YAAY,GAAG,IAAI,CAAC;IAChC;gCAC4B;IAC5B,WAAW,CAAC,GAAG,EAAE,QAAQ,GAAG,IAAI,CAAC;IACjC,UAAU,CAAC,GAAG,EAAE,MAAM,OAAO,GAAG,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,YAAY,EAAE,CAAC;IACvE,WAAW,CAAC,GAAG,EAAE,MAAM,GAAG,WAAW,GAAG,IAAI,CAAC;IAC7C,MAAM,CAAC,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,GAAG,WAAW,CAAC;IAC9C,YAAY,CAAC,GAAG,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,GAAG,WAAW,CAAC;IAC1D,KAAK,IAAI,IAAI,CAAC;CACf"}
|
|
@@ -48,6 +48,74 @@ Codex, and anywhere else skills load. It deliberately holds almost no facts.
|
|
|
48
48
|
- **Compare on effective cost of conformance,** not list price. A model at half
|
|
49
49
|
the price that conforms two-thirds as often is not cheaper.
|
|
50
50
|
|
|
51
|
+
## Setting up a project: key, org, run
|
|
52
|
+
|
|
53
|
+
These blocks are generated from the same source as the quickstart, llms.txt and
|
|
54
|
+
the console's `/setup` page, so they say the same thing everywhere. A human mints
|
|
55
|
+
the key; you do the rest. The SDK reads `MODELRIG_API_KEY` and exports to
|
|
56
|
+
`https://api.modelrig.ai` by default (`MODELRIG_INGEST_URL` overrides it).
|
|
57
|
+
|
|
58
|
+
<!-- BLOCK:KEY -->
|
|
59
|
+
**Mint a key in the org you want the data in.** Sign in at https://app.modelrig.ai, check the organization name in the header badge, and create a key on the API keys page. The `rig_sk_` secret is shown once. One key = one org: every row the key exports lands in the organization it was minted in, whichever organization you happen to be viewing in the console. Set it on every service, worker and cron that creates a rig; your provider keys stay where they are.
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
# Wherever your app reads configuration: .env.local, your platform's env vars, a secret manager. Never in code.
|
|
63
|
+
MODELRIG_API_KEY=rig_sk_…
|
|
64
|
+
```
|
|
65
|
+
<!-- /BLOCK:KEY -->
|
|
66
|
+
|
|
67
|
+
<!-- BLOCK:CONFIRM_ORG -->
|
|
68
|
+
**Confirm the org before you trust an empty console.** Run `modelrig doctor` from the same environment your app runs in. It asks GET /v1/whoami which organization the key belongs to, and the name it prints must match the org badge in the console header. If it names a different organization, your data is landing there: switch the console to that organization, or mint a key in the one you meant (https://modelrig.dev/orgs-and-keys.html).
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
npx modelrig doctor
|
|
72
|
+
# [modelrig] key rig_sk_…AbCd → org «Your Org» ✓ (ingest https://api.modelrig.ai, …)
|
|
73
|
+
|
|
74
|
+
# No CLI in that environment? Ask the API directly:
|
|
75
|
+
curl -H "Authorization: Bearer $MODELRIG_API_KEY" https://api.modelrig.ai/v1/whoami
|
|
76
|
+
```
|
|
77
|
+
<!-- /BLOCK:CONFIRM_ORG -->
|
|
78
|
+
|
|
79
|
+
Do this before you trust an empty `/runs`: a valid key from another organization
|
|
80
|
+
exports successfully, just not where your human is looking
|
|
81
|
+
(<https://modelrig.dev/troubleshooting.html>).
|
|
82
|
+
|
|
83
|
+
<!-- BLOCK:RUN_WRAP -->
|
|
84
|
+
**Wrap one pipeline execution in a run.** Every pipeline execution is a run. Wrap each execution in `run.scope` (or `run.start` / `run.end`), name one step per model call with `stepKey`, `artifact.save` the work products, and pass `environment` so the Env column on /runs is filled. Inside the run the SDK stamps `run_id` and `step` on every call. `run_id` is reserved: don't set it yourself.
|
|
85
|
+
|
|
86
|
+
```ts
|
|
87
|
+
import { createRig, loadConfigFromEnv } from "modelrig";
|
|
88
|
+
|
|
89
|
+
const rig = createRig(loadConfigFromEnv());
|
|
90
|
+
|
|
91
|
+
// Every pipeline execution is a run. Inside the scope the SDK records each
|
|
92
|
+
// rig.run / rig.runRaw call as a step and stamps run_id and step on every row.
|
|
93
|
+
// run_id is reserved: don't set it yourself (put your own attempt id in a tag of
|
|
94
|
+
// your own, such as pipeline_attempt_id).
|
|
95
|
+
await rig.artifacts.run.scope(
|
|
96
|
+
{ pipeline: "support", episodeKey: ticketId, environment: process.env.NODE_ENV ?? "development" },
|
|
97
|
+
async () => {
|
|
98
|
+
const result = await rig.run("example.support_summarize", {
|
|
99
|
+
input: { ticket, product, priorContext },
|
|
100
|
+
tags: { subject: customer, feature: "summarize" },
|
|
101
|
+
stepKey: "summarize", // names this step on /runs
|
|
102
|
+
});
|
|
103
|
+
rig.artifacts.artifact.save(result.output, { name: "summary", type: "step_output", stepKey: "summarize" });
|
|
104
|
+
},
|
|
105
|
+
);
|
|
106
|
+
|
|
107
|
+
await rig.close(); // flushes the exporter — call once at shutdown
|
|
108
|
+
```
|
|
109
|
+
<!-- /BLOCK:RUN_WRAP -->
|
|
110
|
+
|
|
111
|
+
<!-- BLOCK:NOT_CAPTURED -->
|
|
112
|
+
**What ModelRig does not see.** ModelRig records the calls your code makes through the SDK (`rig.run`, `rig.runBundle`, `rig.runRaw`, `rig.runRawStream`) or the gateway, and nothing else. Five things stay invisible or unattached: (1) direct provider SDK calls, with or without a JSON schema; (2) streams you read from a provider client yourself; (3) a fallback your code makes after the raw lane refuses a `responseSchema` it cannot enforce (deepseek, deepinfra, fireworks and zai fail closed with `invariant_violation`); (4) calls made outside `run.scope` / `run.start`, which land on /inferences but on no run; (5) embeddings. Details and the console tells: https://modelrig.dev/not-captured.html.
|
|
113
|
+
<!-- /BLOCK:NOT_CAPTURED -->
|
|
114
|
+
|
|
115
|
+
Which lane fits a codebase (observe, the `runRaw` seam, or named routes):
|
|
116
|
+
<https://modelrig.dev/choosing-a-lane.html>. Hosted workers:
|
|
117
|
+
<https://modelrig.dev/deploying.html>.
|
|
118
|
+
|
|
51
119
|
## Just want to see your runs first? (observe, no migration)
|
|
52
120
|
|
|
53
121
|
Before migrating anything, you can just **watch** an existing pipeline. ModelRig
|
|
@@ -82,11 +150,11 @@ never the skill's job.
|
|
|
82
150
|
**Runs are the standard path, not an appendix.** Every pipeline execution should
|
|
83
151
|
be a **run**: wrap it in `rig.artifacts.run.start` / `run.scope`, name one step
|
|
84
152
|
per model-call family (pass `stepKey`), `artifact.save` the work products, and tag
|
|
85
|
-
`subject` / `feature`. Inside a run context the SDK auto-stamps `tags.run_id`
|
|
86
|
-
you, so the run fills `/runs` **and** `/projects`; acceptance is
|
|
87
|
-
on `/runs` with its steps.
|
|
88
|
-
|
|
89
|
-
|
|
153
|
+
`subject` / `feature`. Inside a run context the SDK auto-stamps `tags.run_id` and
|
|
154
|
+
`tags.step` for you, so the run fills `/runs` **and** `/projects`; acceptance is
|
|
155
|
+
the run appearing on `/runs` with its steps. `run_id` is reserved: never set it
|
|
156
|
+
yourself (use a key of your own, such as `pipeline_attempt_id`, for the
|
|
157
|
+
pipeline's attempt id); name steps with `stepKey`. This is Tier 1 and it is **not
|
|
90
158
|
optional for a raw-lane (Lane-B) seam** — a pipeline that reaches the model
|
|
91
159
|
through `rig.runRaw` without a `run.start` around it records inferences but no
|
|
92
160
|
run. The full API is the reference at
|