@dbx-tools/model 0.1.112 → 0.3.2

package/README.md

@@ -1,105 +1,165 @@
 # @dbx-tools/model
 
-Workspace-aware model selection for Databricks Model Serving: list a
-workspace's `/serving-endpoints` (cached), fuzzy-match loose names like
-`"claude sonnet"` to real endpoint ids, rank endpoints by capability
-class, and resolve a single usable model id with an offline fallback
-floor.
+Workspace-aware Databricks Model Serving selection.
+
+Import this package when server-side code needs to turn a loose model request
+like `"claude sonnet"` or `"chat-fast"` into a concrete serving endpoint in the
+current workspace. It lists `/serving-endpoints`, caches and enriches the
+catalogue, classifies endpoints by capability, fuzzy-matches names, and falls
+back to a small static floor when the live catalogue is unavailable.
+
+Browser-safe request/result schemas and endpoint classification types live in
+[`@dbx-tools/shared-model`](../../shared/model).
+
+Key features:
+
+- Lists Databricks Model Serving endpoints through the SDK and normalizes them
+  into a stable summary shape.
+- Classifies endpoints into chat-thinking, chat-balanced, chat-fast, and
+  embedding classes using Foundation Model API scores and family heuristics.
+- Resolves loose user input such as `"sonnet"` or `"chat fast"` to a concrete
+  endpoint id.
+- Supports class ceilings so callers can ask for a capability band without
+  accidentally escalating to a larger model.
+- Caches enriched catalogues per workspace host through AppKit cache utilities.
+- Provides a small static fallback floor for local tools and degraded workspace
+  access.
+
+## Why Not Just AppKit Serving?
+
+Native AppKit's Model Serving plugin is the right choice when you already know
+the endpoint alias you want. It gives you authenticated invoke/stream routes,
+OBO execution, generated endpoint types, request-body filtering, and frontend
+hooks.
+
+Use this package before or beside that layer when the hard part is choosing the
+endpoint:
+
+- resolve loose human input such as `"sonnet"` or `"fast"` against the live
+  workspace catalogue;
+- group endpoints into capability classes like `chat-thinking`, `chat-balanced`,
+  `chat-fast`, and `embedding`;
+- enforce class ceilings so a caller can degrade to smaller models without
+  escalating to a larger one;
+- build model pickers and debug routes from a cached, enriched endpoint list;
+- keep local agents and CLIs working with a static fallback when catalogue
+  access is unavailable.
+
+## Select One Model
 
-This is the server-side package (it holds a `WorkspaceClient` and
-AppKit's cache). Browser consumers want the pure
-[`@dbx-tools/model-shared`](../model-shared) surface, which this package
-re-exports.
+```ts
+import { WorkspaceClient } from "@databricks/sdk-experimental";
+import { resolve } from "@dbx-tools/model";
+
+const client = new WorkspaceClient({});
+const host = String(await client.config.getHost());
 
-## Install
+const selected = await resolve.selectModel(client, host, {
+  explicit: "claude sonnet",
+});
 
-```bash
-npm install @dbx-tools/model
+console.log(selected.modelId, selected.source);
 ```
 
-Peer: `@databricks/appkit` (provides the cache the listing runs through).
+`selectModel()` is the high-level helper for agents and CLIs. It reads the live
+catalogue, applies fuzzy matching when an explicit string is present, then
+returns a single `modelId` plus a source label explaining why that endpoint won.
 
-## Usage
+The `source` label is useful for logs and debug UIs. It distinguishes explicit
+matches from class-based selection, environment defaults, and fallback results,
+so operators can tell whether a request used the intended model policy.
 
-Hold a `WorkspaceClient` and just want a usable model id? `selectModel`
-lists the catalogue (cached) and resolves in one call. An `explicit`
-name is fuzzy-matched; without one, a `modelClass` ceiling (or the
-operator `fallbacks`, then the static floor) decides.
+## Build A Model Picker
 
 ```ts
-import { WorkspaceClient } from "@databricks/sdk-experimental";
-import { selectModel, searchModels } from "@dbx-tools/model";
+import { resolve } from "@dbx-tools/model";
 
-const client = new WorkspaceClient({});
-const host = (await client.config.getHost()).toString();
+const ranked = await resolve.searchModels(client, host, {
+  search: "opus",
+  modelClass: "chat-thinking",
+  limit: 5,
+});
+```
 
-// Fuzzy name -> real endpoint id (+ how it was reached).
-const { modelId, source } = await selectModel(client, host, {
-  explicit: "claude sonnet",
-}); // -> { modelId: "databricks-claude-sonnet-4-6", source: "fuzzy-match" }
+Use `searchModels()` for UI pickers and debug routes. It returns ranked models
+with match scores and endpoint summaries, using the same fuzzy threshold and
+class ceiling logic as `selectModel()`.
 
-// Best balanced-or-cheaper chat model the workspace actually has.
-const fast = await selectModel(client, host, { modelClass: "chat-balanced" });
+## Work With A Held Catalogue
 
-// Or get the full ranked list (model picker / CLI).
-const ranked = await searchModels(client, host, { search: "opus", limit: 5 });
+When you already have endpoint summaries, use the pure resolver functions from
+`resolve` and `serving` without another workspace call:
+
+```ts
+import { resolve, serving } from "@dbx-tools/model";
+
+const endpoints = await serving.listServingEndpoints(client, host);
+const ranked = resolve.rankModels(endpoints, { search: "sonnet", limit: 3 });
+const picked = resolve.resolveModel(endpoints, {
+  explicit: "claude sonnet",
+  modelClass: "chat-balanced",
+});
 ```
 
-The capability class acts as a **ceiling**: `chat-balanced` can fall to
-`chat-fast` but never escalate to `chat-thinking`. Embedding endpoints
-surface only when `modelClass` is explicitly `embedding`.
+The class acts as a ceiling. `chat-balanced` may fall back to `chat-fast`, but
+will not escalate to `chat-thinking`. Embedding endpoints are considered only
+when the requested class is `embedding`.
+
+## List And Cache Serving Endpoints
+
+```ts
+import { serving } from "@dbx-tools/model";
+
+const endpoints = await serving.listServingEndpoints(client, host, {
+  ttlMs: 5 * 60_000,
+});
+
+const raw = await serving.listServingEndpointsUncached(client);
+await serving.clearServingEndpointsCache(host);
+```
 
-### Pure helpers (no I/O)
+`listServingEndpoints()` uses AppKit's `CacheManager`, enriches endpoints with
+classification and embedding dimensions, and keys the cache by workspace host.
+`listServingEndpointsUncached()` is useful for simple scripts that only need the
+SDK response and do not want a cache dependency.
 
-If you already hold the endpoint list (e.g. from a cached `/models`
-response), use the pure functions directly:
+## Fuzzy Resolve Endpoint Names
 
 ```ts
-import { resolveModelId, rankModels, resolveModel } from "@dbx-tools/model";
+const matches = serving.searchServingEndpoints("claude sonnet", endpoints, {
+  threshold: 0.35,
+});
 
-resolveModelId("claude sonnet", endpoints); // single best fuzzy match
-rankModels(endpoints, { search: "opus", limit: 3 }); // ranked list
-resolveModel(endpoints, { modelClass: "chat-fast" }); // single id + source
+const endpointName = serving.resolveModelId("sonnet", endpoints);
 ```
 
-### Listing the catalogue directly
+Fuzzy matching is intentionally a server concern because it depends on the live
+workspace catalogue and may re-list on misses. Disable it in callers that require
+exact endpoint ids.
 
-`selectModel` / `searchModels` list endpoints for you, but you can also
-reach the catalogue functions:
+## Use Static Fallbacks
 
 ```ts
-import {
-  listServingEndpoints,
-  listServingEndpointsUncached,
-  searchServingEndpoints,
-  clearServingEndpointsCache,
-} from "@dbx-tools/model";
-
-// Cached + enriched (model-class stamp, embedding dimensions) through
-// AppKit's CacheManager. Keyed by `host`.
-const endpoints = await listServingEndpoints(client, host);
-
-// One-shot, dependency-light: straight from the SDK, no cache and no
-// enrichment. For CLIs / non-AppKit contexts that only need names.
-const raw = await listServingEndpointsUncached(client);
-
-// Fuzzy-rank a held list by name (the core resolveModelId builds on).
-searchServingEndpoints("claude sonnet", endpoints); // ScoredEndpoint[]
-
-// Force-evict one host's cache entry (or all when omitted).
-await clearServingEndpointsCache(host);
+import { classes, fallback } from "@dbx-tools/model";
+import { model } from "@dbx-tools/shared-model";
+
+const cls = classes.parseModelClass("chat-fast") ?? model.ModelClass.ChatFast;
+const modelId = fallback.modelForClass(cls);
 ```
 
-## How resolution works
+The fallback floor gives agents and local scripts a stable answer when a
+workspace cannot list endpoints. Prefer live catalogue resolution for production
+policy decisions; fallbacks are a last resort.
+
+## Modules
 
-1. **Explicit ask** - fuzzy-matched within the optional class ceiling
-   (or returned verbatim when `fuzzy: false`).
-2. **No explicit ask** - the first operator-pinned `fallbacks` entry
-   that exists in the live catalogue wins first, then the ranked live
-   catalogue, then a small static `FALLBACK_MODEL_IDS` floor when the
-   catalogue yields nothing in range (so a model id is always returned
-   even offline).
+- `resolve` - high-level `selectModel`, ranked search, and catalogue-held
+  resolver functions.
+- `serving` - Databricks serving-endpoint listing, cache management, fuzzy
+  search, and endpoint-id resolution.
+- `classes` - model-class parsing, ordering, and class-ceiling helpers.
+- `fallback` - static fallback model ids per class.
 
-Capability bands are derived from the live workspace catalogue (the
-Foundation Model API quality/speed/cost scores), not a hand-maintained
-table - see `@dbx-tools/model-shared` for the classifier.
+The AppKit-Mastra integration uses this package through
+[`@dbx-tools/appkit-mastra`](../appkit-mastra); the local OpenAI-compatible
+gateway uses it through [`@dbx-tools/model-proxy`](../../cli/model-proxy).

package/index.ts

@@ -0,0 +1,10 @@
+// GENERATED by projen watch - DO NOT EDIT.
+// Regenerated from the exporting modules in ./src.
+// Hand edits are overwritten on the next watch; this file is read-only.
+
+export * as classes from "./src/classes";
+export * as fallback from "./src/fallback";
+export * as resolve from "./src/resolve";
+export * as serving from "./src/serving";
+export type { ResolveModelInput, ResolvedModelSelection, SelectModelInput, SearchModelsInput } from "./src/resolve";
+export type { WorkspaceClientLike, ListServingEndpointsOptions, ResolvedModel, ResolveModelOptions, ScoredEndpoint } from "./src/serving";

package/package.json

@@ -1,28 +1,49 @@
 {
   "name": "@dbx-tools/model",
-  "version": "0.1.112",
-  "dependencies": {
-    "@dbx-tools/model-shared": "0.1.112",
-    "@dbx-tools/shared": "0.1.112",
-    "fuse.js": "^7.4.2"
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/reggie-db/dbx-tools.git",
+    "directory": "workspaces/node/model"
   },
-  "exports": {
-    ".": {
-      "types": "./dist/index.d.ts",
-      "default": "./dist/index.js"
-    }
+  "devDependencies": {
+    "@types/node": "^24.6.0",
+    "tsx": "^4.23.0",
+    "typescript": "^5.9.3"
   },
-  "peerDependencies": {
-    "@databricks/appkit": "^0.43.0"
+  "dependencies": {
+    "@databricks/appkit": "^0.43.0",
+    "fuse.js": "^7.4.2",
+    "@dbx-tools/appkit": "0.3.2",
+    "@dbx-tools/shared-core": "0.3.2",
+    "@dbx-tools/shared-model": "0.3.2"
   },
-  "type": "module",
-  "main": "./dist/index.js",
-  "types": "./dist/index.d.ts",
-  "files": [
-    "dist"
-  ],
-  "license": "Apache-2.0",
+  "main": "index.ts",
+  "license": "UNLICENSED",
   "publishConfig": {
     "access": "public"
+  },
+  "version": "0.3.2",
+  "types": "index.ts",
+  "type": "module",
+  "exports": {
+    ".": "./index.ts",
+    "./package.json": "./package.json"
+  },
+  "dbxToolsConfig": {
+    "tags": [
+      "node"
+    ]
+  },
+  "//": "~~ Generated by projen. To modify, edit .projenrc.js and run \"pnpm exec projen\".",
+  "scripts": {
+    "build": "projen build",
+    "compile": "projen compile",
+    "default": "projen default",
+    "package": "projen package",
+    "post-compile": "projen post-compile",
+    "pre-compile": "projen pre-compile",
+    "test": "projen test",
+    "watch": "projen watch",
+    "projen": "projen"
   }
-}
+}

package/src/classes.ts

@@ -0,0 +1,73 @@
+/**
+ * Model-class ordering for the model service.
+ *
+ * `@dbx-tools/shared-model` declares the {@link ModelClass} values and their
+ * schema; the *behavior* over those values - the chat capability order, the
+ * "this class and below" ceiling, and coercing loose request input to a class -
+ * lives here in the service so the shared wire-format surface stays purely
+ * declarative.
+ */
+
+import { model } from "@dbx-tools/shared-model";
+
+type ModelClass = model.ModelClass;
+const { ModelClass, ModelClassSchema } = model;
+
+/**
+ * Chat capability ladder in descending order - most capable
+ * ({@link ModelClass.ChatThinking}) first, least ({@link ModelClass.ChatFast})
+ * last. {@link ModelClass.Embedding} is a separate modality and is deliberately
+ * absent: the ceiling never spans it. This is the order used to filter "this
+ * class and below" and to break ranking ties toward the more capable model.
+ */
+export const CHAT_CLASS_ORDER: readonly ModelClass[] = [
+  ModelClass.ChatThinking,
+  ModelClass.ChatBalanced,
+  ModelClass.ChatFast,
+];
+
+/**
+ * Every class in display order: the chat ladder followed by
+ * {@link ModelClass.Embedding}. Used when flattening a full classification
+ * (e.g. stamping the class onto each cached endpoint).
+ */
+export const MODEL_CLASS_ORDER: readonly ModelClass[] = [...CHAT_CLASS_ORDER, ModelClass.Embedding];
+
+/** Whether `cls` is one of the chat capability bands (vs. embedding). */
+export function isChatClass(cls: ModelClass): boolean {
+  return CHAT_CLASS_ORDER.includes(cls);
+}
+
+/**
+ * Coerce an arbitrary value (query string, header, body field) to a
+ * {@link ModelClass}, returning `null` when it isn't a known class. Lets a
+ * route accept a class request without throwing on junk input.
+ *
+ * Matches the full slug first (`"chat-fast"`, `"embedding"`), then - for a bare
+ * chat band - retries with a `chat-` prefix, so the shorthand `"thinking"` /
+ * `"balanced"` / `"fast"` resolves to the corresponding `chat-*` class.
+ */
+export function parseModelClass(value: unknown): ModelClass | null {
+  const exact = ModelClassSchema.safeParse(value);
+  if (exact.success) return exact.data;
+  const prefixed = ModelClassSchema.safeParse(`chat-${value}`);
+  return prefixed.success ? prefixed.data : null;
+}
+
+/**
+ * Classes at or below `cls` in chat capability: the class itself plus every
+ * less-capable chat band, in {@link CHAT_CLASS_ORDER}. Treats a chat `cls` as a
+ * ceiling - requesting {@link ModelClass.ChatBalanced} yields
+ * `[ChatBalanced, ChatFast]`, never {@link ModelClass.ChatThinking} - so a
+ * class ask can degrade downward to a smaller chat model but never escalate to
+ * a larger one.
+ *
+ * {@link ModelClass.Embedding} is its own modality, not a rung on the chat
+ * ladder, so it yields just `[Embedding]`. An unrecognized class yields the
+ * full chat ladder.
+ */
+export function classesAtOrBelow(cls: ModelClass): ModelClass[] {
+  if (cls === ModelClass.Embedding) return [ModelClass.Embedding];
+  const index = CHAT_CLASS_ORDER.indexOf(cls);
+  return index < 0 ? [...CHAT_CLASS_ORDER] : [...CHAT_CLASS_ORDER.slice(index)];
+}

package/src/fallback.ts

@@ -0,0 +1,73 @@
+/**
+ * Server-side offline fallback opinion for model selection.
+ *
+ * When the live `/serving-endpoints` catalogue can't be read at all - no user
+ * token, the service principal can't list, or the workspace is unreachable -
+ * the resolver still has to name *some* endpoint. This module holds that floor:
+ * a small, hard-coded set of well-known Foundation Model API endpoint names,
+ * each bucketed into a chat {@link ModelClass} by the shared
+ * {@link classify.classifyByFamily} heuristic (no classes are hard-coded here)
+ * and ordered best-first.
+ *
+ * This is deliberately *server-only*. A browser client never talks to
+ * Databricks directly - it always goes through this server - so it has nothing
+ * to fall back to and must not assume a stale, baked-in model list; it consumes
+ * the live `/models` response instead. The pure classifier
+ * ({@link classify.classifyEndpoints}) is what the client shares.
+ */
+
+import { classify, model, type FamilyClass } from "@dbx-tools/shared-model";
+
+type ModelClass = model.ModelClass;
+const { ModelClass } = model;
+
+/**
+ * Small, last-resort set of well-known Foundation Model API endpoint names,
+ * ordered best-first within each class by {@link classify.classifyByFamily}.
+ * Used only as the floor when the live `/serving-endpoints` catalogue can't be
+ * read at resolve time; the live, score-driven classification supersedes it
+ * whenever the workspace listing is available. Classes are not hard-coded here
+ * - each name is classified by family heuristic.
+ */
+const FALLBACK_MODEL_NAMES: readonly string[] = [
+  "databricks-claude-opus-4-8",
+  "databricks-gpt-5-5-pro",
+  "databricks-gemini-3-1-pro",
+  "databricks-claude-sonnet-4-6",
+  "databricks-gpt-5-5",
+  "databricks-meta-llama-3-3-70b-instruct",
+  "databricks-claude-haiku-4-5",
+  "databricks-gpt-5-nano",
+  "databricks-meta-llama-3-1-8b-instruct",
+];
+
+/**
+ * Static fallback model ids for a chat class, drawn from the small built-in
+ * {@link FALLBACK_MODEL_NAMES} list and ordered best-first by family rank. Sync
+ * and workspace-independent: this is the *fallback opinion* used to seed default
+ * lists or when the live catalogue is unreachable - live resolution prefers
+ * {@link classify.classifyEndpoints}. Returns `[]` for
+ * {@link ModelClass.Embedding} (the floor is chat-only).
+ */
+export function modelsForClass(cls: ModelClass): readonly string[] {
+  return FALLBACK_MODEL_NAMES.map((name) => ({ name, c: classify.classifyByFamily(name) }))
+    .filter((x): x is { name: string; c: FamilyClass } => x.c !== null && x.c.class === cls)
+    .sort((a, b) => b.c.rank - a.c.rank)
+    .map((x) => x.name);
+}
+
+/** Top static fallback model id for a chat class. */
+export function modelForClass(cls: ModelClass): string {
+  return modelsForClass(cls)[0]!;
+}
+
+/**
+ * Priority-ordered fallback chain (ChatThinking -> ChatBalanced -> ChatFast)
+ * over the small built-in list. The floor walked at resolve time when no agent
+ * / plugin / env / request model is set *and* the live catalogue yields nothing.
+ */
+export const FALLBACK_MODEL_IDS: readonly string[] = [
+  ...modelsForClass(ModelClass.ChatThinking),
+  ...modelsForClass(ModelClass.ChatBalanced),
+  ...modelsForClass(ModelClass.ChatFast),
+];