@databricks/appkit 0.71.0 → 0.73.0

package/docs/api/appkit/Function.matches.md

@@ -0,0 +1,18 @@
+# Function: matches()
+
+```ts
+function matches(pattern: RegExp): Matcher;
+
+```
+
+Passes when the value matches `pattern`.
+
+## Parameters[​](#parameters "Direct link to Parameters")
+
+| Parameter | Type     |
+| --------- | -------- |
+| `pattern` | `RegExp` |
+
+## Returns[​](#returns "Direct link to Returns")
+
+[`Matcher`](./docs/api/appkit/TypeAlias.Matcher.md)

package/docs/api/appkit/Function.normalizeHost.md

@@ -0,0 +1,18 @@
+# Function: normalizeHost()
+
+```ts
+function normalizeHost(raw: string): string;
+
+```
+
+Ensure the host has a scheme (Databricks env often lacks `https://`).
+
+## Parameters[​](#parameters "Direct link to Parameters")
+
+| Parameter | Type     |
+| --------- | -------- |
+| `raw`     | `string` |
+
+## Returns[​](#returns "Direct link to Returns")
+
+`string`

package/docs/api/appkit/Function.readEvalDataset.md

@@ -0,0 +1,21 @@
+# Function: readEvalDataset()
+
+```ts
+function readEvalDataset(client: WorkspaceClient, options: ReadEvalDatasetOptions): Promise<DatasetRow[]>;
+
+```
+
+Read a Databricks managed evaluation dataset (a Unity Catalog table with `inputs`/`expectations` columns) into rows, over the public SQL Statement Execution API. Reuses SQLWarehouseConnector for submit/poll/transform — its result transform already JSON-parses string columns into objects, so `inputs`/`expectations` come back as records whether the table stores them as JSON strings or structs.
+
+The Python `mlflow.genai.datasets` API needs a Spark session (no TS equivalent), so we read the backing table directly.
+
+## Parameters[​](#parameters "Direct link to Parameters")
+
+| Parameter | Type                                                                                    |
+| --------- | --------------------------------------------------------------------------------------- |
+| `client`  | [`WorkspaceClient`](./docs/api/appkit/Interface.WorkspaceClient.md)               |
+| `options` | [`ReadEvalDatasetOptions`](./docs/api/appkit/Interface.ReadEvalDatasetOptions.md) |
+
+## Returns[​](#returns "Direct link to Returns")
+
+`Promise`<[`DatasetRow`](./docs/api/appkit/Interface.DatasetRow.md)\[]>

package/docs/api/appkit/Function.reportToMlflow.md

@@ -0,0 +1,23 @@
+# Function: reportToMlflow()
+
+```ts
+function reportToMlflow(
+   client: MlflowClient, 
+   results: EvalResult[], 
+sqlWarehouseId?: string): Promise<ReportOutcome>;
+
+```
+
+Write one pass/fail assessment per eval result to the Databricks MLflow REST API. Never throws — failures are collected so the run still reports.
+
+## Parameters[​](#parameters "Direct link to Parameters")
+
+| Parameter         | Type                                                               |
+| ----------------- | ------------------------------------------------------------------ |
+| `client`          | [`MlflowClient`](./docs/api/appkit/Class.MlflowClient.md)    |
+| `results`         | [`EvalResult`](./docs/api/appkit/Interface.EvalResult.md)\[] |
+| `sqlWarehouseId?` | `string`                                                           |
+
+## Returns[​](#returns "Direct link to Returns")
+
+`Promise`<[`ReportOutcome`](./docs/api/appkit/Interface.ReportOutcome.md)>

package/docs/api/appkit/Function.resolveDatabricksAuth.md

@@ -0,0 +1,16 @@
+# Function: resolveDatabricksAuth()
+
+```ts
+function resolveDatabricksAuth(options: ResolveDatabricksAuthOptions): Promise<DatabricksAuth | undefined>;
+
+```
+
+## Parameters[​](#parameters "Direct link to Parameters")
+
+| Parameter | Type                                                                                                |
+| --------- | --------------------------------------------------------------------------------------------------- |
+| `options` | [`ResolveDatabricksAuthOptions`](./docs/api/appkit/Interface.ResolveDatabricksAuthOptions.md) |
+
+## Returns[​](#returns "Direct link to Returns")
+
+`Promise`<[`DatabricksAuth`](./docs/api/appkit/Interface.DatabricksAuth.md) | `undefined`>

package/docs/api/appkit/Function.resolveWorkspaceClient.md

@@ -0,0 +1,18 @@
+# Function: resolveWorkspaceClient()
+
+```ts
+function resolveWorkspaceClient(options: ResolveDatabricksAuthOptions): WorkspaceClient | undefined;
+
+```
+
+Construct a Databricks `WorkspaceClient` for the eval runner — the object the SDK-backed connectors (e.g. `SQLWarehouseConnector`) take. An explicit host+token builds a PAT client; otherwise the profile (or ambient config) is used and the SDK resolves credentials, minting OAuth as needed. Returns `undefined` if construction throws (missing/invalid config).
+
+## Parameters[​](#parameters "Direct link to Parameters")
+
+| Parameter | Type                                                                                                |
+| --------- | --------------------------------------------------------------------------------------------------- |
+| `options` | [`ResolveDatabricksAuthOptions`](./docs/api/appkit/Interface.ResolveDatabricksAuthOptions.md) |
+
+## Returns[​](#returns "Direct link to Returns")
+
+[`WorkspaceClient`](./docs/api/appkit/Interface.WorkspaceClient.md) | `undefined`

package/docs/api/appkit/Function.runEval.md

@@ -0,0 +1,19 @@
+# Function: runEval()
+
+```ts
+function runEval(def: EvalDefinition, options: RunEvalOptions): Promise<EvalResult>;
+
+```
+
+Run a single eval against a driver. Never throws for assertion or agent failures — those become a non-passing [EvalResult](./docs/api/appkit/Interface.EvalResult.md). Only a malformed eval definition surfaces as `result.error`.
+
+## Parameters[​](#parameters "Direct link to Parameters")
+
+| Parameter | Type                                                                    |
+| --------- | ----------------------------------------------------------------------- |
+| `def`     | [`EvalDefinition`](./docs/api/appkit/Interface.EvalDefinition.md) |
+| `options` | [`RunEvalOptions`](./docs/api/appkit/Interface.RunEvalOptions.md) |
+
+## Returns[​](#returns "Direct link to Returns")
+
+`Promise`<[`EvalResult`](./docs/api/appkit/Interface.EvalResult.md)>

package/docs/api/appkit/Function.runEvalsInDir.md

@@ -0,0 +1,18 @@
+# Function: runEvalsInDir()
+
+```ts
+function runEvalsInDir(options: RunEvalsOptions): Promise<EvalRunSummary>;
+
+```
+
+Discover, load, and run every eval under each agent's `evals/` dir, driving the agents on a running app. Never throws for an individual eval — load/run failures become non-passing [EvalResult](./docs/api/appkit/Interface.EvalResult.md)s.
+
+## Parameters[​](#parameters "Direct link to Parameters")
+
+| Parameter | Type                                                                      |
+| --------- | ------------------------------------------------------------------------- |
+| `options` | [`RunEvalsOptions`](./docs/api/appkit/Interface.RunEvalsOptions.md) |
+
+## Returns[​](#returns "Direct link to Returns")
+
+`Promise`<[`EvalRunSummary`](./docs/api/appkit/Interface.EvalRunSummary.md)>

package/docs/api/appkit/Function.summarize.md

@@ -0,0 +1,16 @@
+# Function: summarize()
+
+```ts
+function summarize(results: EvalResult[]): EvalSummary;
+
+```
+
+## Parameters[​](#parameters "Direct link to Parameters")
+
+| Parameter | Type                                                               |
+| --------- | ------------------------------------------------------------------ |
+| `results` | [`EvalResult`](./docs/api/appkit/Interface.EvalResult.md)\[] |
+
+## Returns[​](#returns "Direct link to Returns")
+
+[`EvalSummary`](./docs/api/appkit/Interface.EvalSummary.md)

package/docs/api/appkit/Interface.AssertionHandle.md

@@ -0,0 +1,54 @@
+# Interface: AssertionHandle
+
+Chainable handle returned by every assertion to control its severity. Mirrors eve: assertions are gates by default; `.soft()` demotes to a tracked metric; `.atLeast(n)` is a soft, score-thresholded assertion.
+
+## Methods[​](#methods "Direct link to Methods")
+
+### atLeast()[​](#atleast "Direct link to atLeast()")
+
+```ts
+atLeast(threshold: number): AssertionHandle;
+
+```
+
+Set the pass threshold for a scored assertion: it passes only when the score is at least `threshold`. Keeps the current severity (gate unless also chained with `.soft()`).
+
+#### Parameters[​](#parameters "Direct link to Parameters")
+
+| Parameter   | Type     |
+| ----------- | -------- |
+| `threshold` | `number` |
+
+#### Returns[​](#returns "Direct link to Returns")
+
+`AssertionHandle`
+
+***
+
+### gate()[​](#gate "Direct link to gate()")
+
+```ts
+gate(): AssertionHandle;
+
+```
+
+Promote to a hard gate — failure fails the eval (non-zero exit).
+
+#### Returns[​](#returns-1 "Direct link to Returns")
+
+`AssertionHandle`
+
+***
+
+### soft()[​](#soft "Direct link to soft()")
+
+```ts
+soft(): AssertionHandle;
+
+```
+
+Demote to a tracked metric — doesn't fail unless running with `strict`.
+
+#### Returns[​](#returns-2 "Direct link to Returns")
+
+`AssertionHandle`

package/docs/api/appkit/Interface.AssertionResult.md

@@ -0,0 +1,48 @@
+# Interface: AssertionResult
+
+A single recorded assertion outcome.
+
+## Properties[​](#properties "Direct link to Properties")
+
+### detail?[​](#detail "Direct link to detail?")
+
+```ts
+optional detail: string;
+
+```
+
+***
+
+### label[​](#label "Direct link to label")
+
+```ts
+label: string;
+
+```
+
+***
+
+### pass[​](#pass "Direct link to pass")
+
+```ts
+pass: boolean;
+
+```
+
+***
+
+### score?[​](#score "Direct link to score?")
+
+```ts
+optional score: number;
+
+```
+
+***
+
+### severity[​](#severity "Direct link to severity")
+
+```ts
+severity: Severity;
+
+```

package/docs/api/appkit/Interface.Assessment.md

@@ -0,0 +1,83 @@
+# Interface: Assessment
+
+A Feedback assessment in the MLflow REST proto-JSON shape.
+
+## Properties[​](#properties "Direct link to Properties")
+
+### assessment\_name[​](#assessment_name "Direct link to assessment_name")
+
+```ts
+assessment_name: string;
+
+```
+
+***
+
+### feedback[​](#feedback "Direct link to feedback")
+
+```ts
+feedback: {
+  value: unknown;
+};
+
+```
+
+#### value[​](#value "Direct link to value")
+
+```ts
+value: unknown;
+
+```
+
+***
+
+### metadata?[​](#metadata "Direct link to metadata?")
+
+```ts
+optional metadata: Record<string, string>;
+
+```
+
+***
+
+### rationale?[​](#rationale "Direct link to rationale?")
+
+```ts
+optional rationale: string;
+
+```
+
+***
+
+### source[​](#source "Direct link to source")
+
+```ts
+source: {
+  source_id: string;
+  source_type: "CODE" | "HUMAN" | "LLM_JUDGE";
+};
+
+```
+
+#### source\_id[​](#source_id "Direct link to source_id")
+
+```ts
+source_id: string;
+
+```
+
+#### source\_type[​](#source_type "Direct link to source_type")
+
+```ts
+source_type: "CODE" | "HUMAN" | "LLM_JUDGE";
+
+```
+
+***
+
+### trace\_id[​](#trace_id "Direct link to trace_id")
+
+```ts
+trace_id: string;
+
+```

package/docs/api/appkit/Interface.CustomJudgeSpec.md

@@ -0,0 +1,30 @@
+# Interface: CustomJudgeSpec
+
+A custom LLM-judge definition: a prompt template and choice→score mapping.
+
+## Properties[​](#properties "Direct link to Properties")
+
+### choiceScores[​](#choicescores "Direct link to choiceScores")
+
+```ts
+choiceScores: Record<string, number>;
+
+```
+
+***
+
+### name[​](#name "Direct link to name")
+
+```ts
+name: string;
+
+```
+
+***
+
+### promptTemplate[​](#prompttemplate "Direct link to promptTemplate")
+
+```ts
+promptTemplate: string;
+
+```

package/docs/api/appkit/Interface.DatabaseValidationIssue.md

@@ -0,0 +1,21 @@
+# Interface: DatabaseValidationIssue
+
+One rejected field; `path` names public columns, never their values.
+
+## Properties[​](#properties "Direct link to Properties")
+
+### message[​](#message "Direct link to message")
+
+```ts
+readonly message: string;
+
+```
+
+***
+
+### path[​](#path "Direct link to path")
+
+```ts
+readonly path: readonly string[];
+
+```

package/docs/api/appkit/Interface.DatabricksAuth.md

@@ -0,0 +1,21 @@
+# Interface: DatabricksAuth
+
+Resolved Databricks host + bearer token for the eval runner's REST calls.
+
+## Properties[​](#properties "Direct link to Properties")
+
+### host[​](#host "Direct link to host")
+
+```ts
+host: string;
+
+```
+
+***
+
+### token[​](#token "Direct link to token")
+
+```ts
+token: string;
+
+```

package/docs/api/appkit/Interface.DatasetRow.md

@@ -0,0 +1,21 @@
+# Interface: DatasetRow
+
+One row of a managed evaluation dataset. `inputs` are the kwargs passed to the agent for the turn; `expectations` (when present) is the row's ground truth / guidelines. Mirrors the `{inputs, expectations}` shape of `mlflow.genai` datasets and of the Unity Catalog table backing a managed eval dataset.
+
+## Properties[​](#properties "Direct link to Properties")
+
+### expectations?[​](#expectations "Direct link to expectations?")
+
+```ts
+optional expectations: Record<string, unknown>;
+
+```
+
+***
+
+### inputs[​](#inputs "Direct link to inputs")
+
+```ts
+inputs: Record<string, unknown>;
+
+```

package/docs/api/appkit/Interface.DiscoveredEval.md

@@ -0,0 +1,36 @@
+# Interface: DiscoveredEval
+
+An eval file found under `server/agents/<agent>/evals/`.
+
+## Properties[​](#properties "Direct link to Properties")
+
+### agent[​](#agent "Direct link to agent")
+
+```ts
+agent: string;
+
+```
+
+The agent id (the `server/agents/<agent>` directory name).
+
+***
+
+### file[​](#file "Direct link to file")
+
+```ts
+file: string;
+
+```
+
+Absolute path to the `*.eval.ts` file.
+
+***
+
+### id[​](#id "Direct link to id")
+
+```ts
+id: string;
+
+```
+
+Id relative to the agent's evals dir, without `.eval.ts` (e.g. `weather/basic`).

package/docs/api/appkit/Interface.DriveResult.md

@@ -0,0 +1,58 @@
+# Interface: DriveResult
+
+What a driver returns for a single `t.send`.
+
+## Properties[​](#properties "Direct link to Properties")
+
+### reply[​](#reply "Direct link to reply")
+
+```ts
+reply: string;
+
+```
+
+The final assistant message text.
+
+***
+
+### sessionId?[​](#sessionid "Direct link to sessionId?")
+
+```ts
+optional sessionId: string;
+
+```
+
+Thread/session id, when the driver exposes one.
+
+***
+
+### succeeded[​](#succeeded "Direct link to succeeded")
+
+```ts
+succeeded: boolean;
+
+```
+
+Whether the turn completed without an agent/stream error.
+
+***
+
+### toolCalls[​](#toolcalls "Direct link to toolCalls")
+
+```ts
+toolCalls: string[];
+
+```
+
+Names of tools the agent called during the turn.
+
+***
+
+### traceId?[​](#traceid "Direct link to traceId?")
+
+```ts
+optional traceId: string;
+
+```
+
+MLflow trace id for the turn, when tracing is enabled on the app.