@maheidem/model-discovery 0.6.1 → 0.7.1

package/README.md

@@ -4,8 +4,9 @@ Interactive TUI for discovering and managing local AI model endpoints. Works wit
 
 ## Features
 
-- **Auto-detect server type** from headers and model data
+- **Auto-detect server type** from headers and model data (incl. MTPLX's `capability` field)
 - **Read server-reported configuration** — context window, max tokens, reasoning, and vision, with per-model overrides on top
+- **Native-endpoint enrichment** — llama.cpp `/props`, oMLX `/v1/models/status`, and Ollama `/api/tags` + `/api/ps` fill in what the OpenAI layer omits (real context windows, load state, VLM flags), silently best-effort
 - **Auto-detect vision-capable models (VLMs)** — from architecture metadata, llama.cpp `--mmproj` args, or oMLX capabilities
 - **Auto-detect reasoning capability** — from `capabilities`, explicit `reasoning` fields, `--reasoning-budget`, and Qwen model names on oMLX
 - **Auto-detect reasoning format** — oMLX servers get `chat_template_kwargs` thinking support automatically
@@ -26,7 +27,7 @@ Interactive TUI for discovering and managing local AI model endpoints. Works wit
 pi install npm:@maheidem/model-discovery
 
 # Via git
-pi install git:github.com/Maheidem/model-discovery@v0.6.1
+pi install git:github.com/Maheidem/model-discovery@v0.7.0
 ```
 
 ## Usage
@@ -67,6 +68,16 @@ Every field is read from what the server actually reports, first value found win
 
 Detected vision-capable models get `input: ["text", "image"]`, so Pi accepts image input for them. The source defaults and every detection can be corrected per model with **Edit model**.
 
+### Native-endpoint enrichment
+
+For server types that expose richer *native* (non-OpenAI) endpoints, the probe runs one best-effort enrichment pass after the catalogue fetch, filling only what `/v1/models` omitted — explicit values always win:
+
+- **llama.cpp** — `GET /props`: the real runtime context window (`default_generation_settings.n_ctx`) and the authoritative VLM flag (`modalities.vision`)
+- **oMLX** — `GET /v1/models/status`: the effective per-model context window, max output tokens, load state (drives the `[loaded]` flag), and a thinking-capable default
+- **Ollama** — `GET /api/tags` + `GET /api/ps`: the model card's default context length and which models are currently loaded
+
+Enrichment is silent best-effort: a missing or failing native endpoint (or a connection refusal) leaves the catalogue exactly as the OpenAI layer reported it, and the cached catalogue retains the last known-good enrichment for offline fallback.
+
 ### Compatibility settings
 
 The extension attaches `compat` to each registered model (Pi does not merge provider-level compat into individual models):
@@ -196,6 +207,68 @@ If Pi has an `enabledModels` scope, press **Tab** in `/model` to switch from sco
 
 Profiles are retained if a model temporarily disappears during a re-scan.
 
+## Tool-schema repair (local endpoints)
+
+llama.cpp's JSON-schema→grammar converter — the one behind llama.cpp, llama-swap, LM
+Studio, and LiteLLM routes that forward to them — resolves `$ref` pointers **only
+against the root of a tool schema document**. MCP servers that build schemas by nesting
+Pydantic `model_json_schema()` output inside a hand-written parent routinely leave
+`$defs` on an inner node while the `$ref`s inside it stay root-relative:
+
+```jsonc
+{ "properties": { "patch": {
+    "$defs": { "GuidelineMetricInput": { /* ... */ } },        // defs live here
+    "properties": { "metrics": { "items": { "$ref": "#/$defs/GuidelineMetricInput" } } }
+}}}
+```
+
+The pointer resolves against the document root, where `$defs` is not — so the server
+rejects the **entire request**:
+
+```text
+HTTP 400 {"code":400,"message":"JSON schema conversion failed:
+          Error resolving ref #/$defs/GuidelineMetricInput: $defs not in {...}"}
+```
+
+Because the offending tool rides along in every tool list, *every* message in the
+session fails, which looks like a broken endpoint, proxy, or model discovery rather
+than a bad upstream schema.
+
+A second llama.cpp b10612 bug was verified independently: `maxLength: 2000` below
+an array's `items` schema produces `Failed to initialize samplers: failed to parse
+grammar`, while 1999, 2001, and even 65536 all compile. This affected the
+`okto_pulse_move_card` tool even before any `$ref` repair.
+
+For self-hosted endpoints (private/loopback URL, or a detected local engine) the
+extension normalises outgoing tool schemas in `before_provider_request`:
+
+- `$defs` / `definitions` found at any depth are hoisted to a root registry, with
+collisions de-duplicated and local refs rewritten to match;
+- every `$ref` is inlined iteratively, so refs-to-refs collapse;
+- unresolvable or recursive `$ref`s become permissive nodes instead of a hard 400;
+- `$ref` / `$defs` never reach the wire, and annotation siblings (`description`,
+`title`) are preserved;
+- the exact nested-array `maxLength: 2000` failure is sent as 2001. This is the
+least-permissive working neighbour; the MCP server still validates its real 2000 limit.
+
+Cloud APIs and clean payloads are left byte-identical (the payload object's identity is
+returned, no cloning). Repairing a 521-tool catalogue costs ~1.5 ms. Opt out per
+provider with `"repairToolSchemas": false` in `~/.pi/agent/model-discovery.json`, or
+globally with `PI_MODEL_DISCOVERY_NO_SCHEMA_REPAIR=1`. Each distinct repair is logged
+once as `[model-discovery] <provider>: repaired N local tool schema(s): …`.
+
+Live verification against llama-swap v251 → llama.cpp b10612: the raw 521-tool MCP
+catalogue returned HTTP 400; all 16 affected schemas were repaired in flight; the same
+request then returned HTTP 200 with no residual `$ref`/`$defs` on the wire.
+
+Verification:
+
+```bash
+npm test                                              # includes schema-repair.test.ts
+node --experimental-strip-types scripts/live-schema-repair-check.ts http://HOST
+node --experimental-strip-types scripts/bisect-grammar.ts http://HOST MODEL FILTER
+```
+
 ## Offline resilience
 
 Every successful live scan atomically persists the raw model catalogue as the source's last known-good cache. Saved sources are scanned independently and concurrently at startup. If one source is offline, times out, rejects its credentials, or returns a malformed response:

package/index.ts

@@ -40,6 +40,18 @@ import {
 	validateProfileSampling,
 	validateProfileSlug,
 } from "./profiles.ts";
+import {
+	extractModelConfig,
+	fetchModels,
+	redactSecret,
+	type ModelConfig,
+} from "./providers.ts";
+import {
+	describeToolSchemaRepair,
+	isLocalEndpointUrl,
+	repairRequestToolSchemas,
+	type ToolSchemaRepairReport,
+} from "./schema-repair.ts";
 import { existsSync, readFileSync, renameSync, unlinkSync, writeFileSync } from "node:fs";
 import { join } from "node:path";
 import os from "node:os";
@@ -54,7 +66,6 @@ interface ModelOverride {
 	reasoning?: boolean;
 	input?: string[];
 }
-
 interface DiscoveredProvider {
 	name: string;
 	baseUrl: string;
@@ -68,23 +79,18 @@ interface DiscoveredProvider {
 	profileSchemaVersion?: number;
 	cachedModels?: Record<string, unknown>[];
 	compat?: Record<string, unknown>;
+	/**
+	 * Inline $defs/$ref in outgoing tool schemas for this endpoint (default: true for
+	 * local/self-hosted endpoints, where llama.cpp-style grammar converters reject any
+	 * $ref that is not resolvable at the document root). Set false to send verbatim.
+	 */
+	repairToolSchemas?: boolean;
 	/** Last successful live catalogue refresh (legacy name retained in storage). */
 	lastScanned?: number;
 	lastScanAttempt?: number;
 	lastScanError?: string;
 }
 
-interface ModelConfig {
-	id: string;
-	name: string;
-	contextWindow: number | null;
-	maxTokens: number | null;
-	reasoning: boolean | null;
-	input: string[] | null;
-	source: string;
-	loaded?: boolean;
-}
-
 // ---------------------------------------------------------------------------
 // Storage
 // ---------------------------------------------------------------------------
@@ -153,10 +159,6 @@ function errorMessage(error: unknown): string {
 	return error instanceof Error ? error.message : String(error);
 }
 
-function redactSecret(value: string, secret?: string): string {
-	return secret ? value.replaceAll(secret, "[redacted]") : value;
-}
-
 function persistProviderScanState(provider: DiscoveredProvider): void {
 	try {
 		const providers = loadProviders();
@@ -251,204 +253,6 @@ function deleteModelProfileRouting(provider: DiscoveredProvider, modelId: string
 	provider.modelProfileRouting = Object.keys(routing).length > 0 ? routing : undefined;
 }
 
-// ---------------------------------------------------------------------------
-// Server detection & model config extraction (reads real server data)
-// ---------------------------------------------------------------------------
-
-function detectServerType(headers: Headers, models: Record<string, unknown>[]): string {
-	const server = (headers.get("server") ?? "").toLowerCase();
-	const poweredBy = (headers.get("x-powered-by") ?? "").toLowerCase();
-
-	if (server.includes("llama-cpp") || server.includes("llama.cpp")) return "llama.cpp";
-	if (server.includes("ollama")) return "Ollama";
-	if (server.includes("vllm")) return "vLLM";
-	if (server.includes("sglang")) return "SGLang";
-	if (server.includes("lm-studio") || server.includes("lm studio") || server.includes("lmstudio")) return "LM Studio";
-	if (server.includes("omlx") || poweredBy.includes("omlx")) return "oMLX";
-
-	for (const m of models) {
-		const ownedBy = String(m.owned_by ?? "").toLowerCase();
-		if (ownedBy === "omlx") return "oMLX";
-		if (ownedBy === "vllm") return "vLLM";
-		if (ownedBy === "llamacpp") return "llama.cpp";
-	}
-	for (const m of models) {
-		if (String(m.id ?? "").includes(":")) return "Ollama";
-	}
-	return "OpenAI-compatible";
-}
-
-function tryNum(v: unknown): number | null {
-	if (typeof v === "number" && !isNaN(v)) return v;
-	if (typeof v === "string") {
-		const n = parseInt(v, 10);
-		return isNaN(n) ? null : n;
-	}
-	return null;
-}
-
-function parseArgValue(args: string[] | undefined, flag: string): number | null {
-	if (!args) return null;
-	for (let i = 0; i < args.length - 1; i++) {
-		if (args[i] === flag) {
-			const n = parseInt(args[i + 1], 10);
-			return isNaN(n) ? null : n;
-		}
-	}
-	return null;
-}
-
-function parsePresetValue(preset: string | undefined, key: string): number | null {
-	if (!preset) return null;
-	const m = preset.match(new RegExp(`${key}\\s*=\\s*(\\d+)`, "i"));
-	if (m) {
-		const n = parseInt(m[1], 10);
-		return isNaN(n) ? null : n;
-	}
-	return null;
-}
-
-/**
- * Extract model config from whatever the server actually reports.
- * Returns null for any field the server doesn't provide.
- */
-function extractModelConfig(raw: Record<string, unknown>): ModelConfig {
-	const id = String(raw.id ?? "");
-	const name = String(raw.name ?? id);
-	const status = (raw.status && typeof raw.status === "object" ? raw.status : undefined) as
-		| Record<string, unknown>
-		| undefined;
-	const args = status?.args as string[] | undefined;
-	const preset = status?.preset as string | undefined;
-
-	// Context window: standard fields, then llama.cpp args/preset, then loaded meta
-	let contextWindow =
-		tryNum(raw.context_length) ??
-		tryNum(raw.context_window) ??
-		tryNum(raw.max_model_len) ??
-		tryNum(raw.max_context_len) ??
-		tryNum(raw.max_context_length) ??
-		parseArgValue(args, "--ctx-size") ??
-		parsePresetValue(preset, "ctx-size");
-	if (contextWindow === null && raw.meta && typeof raw.meta === "object") {
-		contextWindow = tryNum((raw.meta as Record<string, unknown>).n_ctx);
-	}
-
-	// Max output tokens
-	const maxTokens =
-		tryNum(raw.max_tokens) ??
-		tryNum(raw.max_output_tokens) ??
-		tryNum(raw.max_completion_tokens) ??
-		parseArgValue(args, "--n-predict") ??
-		parsePresetValue(preset, "n-predict");
-
-	// Reasoning
-	let reasoning: boolean | null = null;
-	if (Array.isArray(raw.capabilities)) reasoning = (raw.capabilities as string[]).includes("reasoning");
-	if (reasoning === null && raw.reasoning !== undefined) reasoning = !!raw.reasoning;
-	if (reasoning === null) {
-		const budget = parseArgValue(args, "--reasoning-budget") ?? parsePresetValue(preset, "reasoning-budget");
-		if (budget !== null) reasoning = budget !== 0;
-	}
-
-	// Input modalities
-	let input: string[] | null = null;
-	let hasVision = false;
-
-	// 1. Standard architecture.input_modalities (vLLM, SGLang, etc.)
-	if (raw.architecture && typeof raw.architecture === "object") {
-		const arch = raw.architecture as Record<string, unknown>;
-		const modalities = arch.input_modalities as string[] | undefined;
-		if (Array.isArray(modalities) && modalities.length > 0) {
-			input = [];
-			for (const m of modalities) {
-				const l = m.toLowerCase();
-				if (l.includes("text") && !input.includes("text")) input.push("text");
-				if ((l.includes("image") || l.includes("vision")) && !input.includes("image")) {
-					input.push("image");
-					hasVision = true;
-				}
-			}
-		}
-		// Also check for vision-specific architecture keys
-		if (!hasVision && (arch.vision_config || arch.vision_model || arch.mm_proj || arch.multi_modal_projector)) {
-			hasVision = true;
-		}
-	}
-
-	// 2. Direct input array on the model object
-	if (!input && Array.isArray(raw.input)) {
-		input = raw.input as string[];
-		if (input.includes("image")) hasVision = true;
-	}
-
-	// 3. llama.cpp: --mmproj flag in args or preset (multimodal projector file)
-	if (!hasVision && args) {
-		for (const a of args) {
-			if (a.startsWith("--mmproj") || a.startsWith("--vision")) {
-				hasVision = true;
-				break;
-			}
-		}
-	}
-	if (!hasVision && preset) {
-		if (/mmproj|vision/i.test(preset)) {
-			hasVision = true;
-		}
-	}
-
-	// 4. oMLX: check for vision-specific capabilities or model tags
-	if (!hasVision && Array.isArray(raw.capabilities)) {
-		const caps = (raw.capabilities as string[]).map((c: string) => c.toLowerCase());
-		if (caps.some((c: string) => c.includes("vision") || c.includes("image") || c.includes("multimodal"))) {
-			hasVision = true;
-		}
-	}
-
-	// 5. Build final input array — always include "text", add "image" if vision detected
-	if (hasVision) {
-		input = input && input.includes("image") ? input : ["text", "image"];
-	} else if (!input) {
-		input = ["text"];
-	} else if (!input.includes("text")) {
-		input.unshift("text");
-	}
-
-	const loaded = status?.value === "loaded" ? true : status?.value === "unloaded" ? false : undefined;
-	const source = String(raw.source ?? (status ? "server args" : "api"));
-
-	return { id, name, contextWindow, maxTokens, reasoning, input, source, loaded };
-}
-
-async function fetchModels(
-	baseUrl: string,
-	apiKey?: string,
-	signal?: AbortSignal,
-): Promise<{ models: Record<string, unknown>[]; serverType: string }> {
-	const url = baseUrl.replace(/\/+$/, "") + "/v1/models";
-	const headers: Record<string, string> = { Accept: "application/json" };
-	if (apiKey) headers.Authorization = `Bearer ${apiKey}`;
-
-	const response = await fetch(url, { headers, signal });
-	if (!response.ok) {
-		const body = await response.text().catch(() => "");
-		throw new Error(`HTTP ${response.status}: ${redactSecret(body.slice(0, 200), apiKey)}`);
-	}
-	const data = (await response.json()) as Record<string, unknown>;
-	if (!data || typeof data !== "object" || !Array.isArray(data.data)) {
-		throw new Error("Invalid /v1/models response: expected a data array.");
-	}
-	const models = data.data.filter(
-		(model): model is Record<string, unknown> =>
-			!!model && typeof model === "object" && typeof (model as Record<string, unknown>).id === "string" &&
-			(model as Record<string, unknown>).id !== "",
-	);
-	if (models.length !== data.data.length) {
-		throw new Error("Invalid /v1/models response: every model must have a non-empty string id.");
-	}
-	return { models, serverType: detectServerType(response.headers, models) };
-}
-
 function generateProviderName(url: string): string {
 	try {
 		const u = new URL(url);
@@ -475,8 +279,36 @@ export default async function (pi: ExtensionAPI) {
 	};
 	const thinkingRoutes = new Map<string, RuntimeThinkingRoutes>();
 	const fixedProfileLabels = new Map<string, string>();
+	/** Providers whose outgoing tool schemas get local grammar compatibility repair. */
+	const schemaRepairProviders = new Set<string>();
+	/** Repair notices already surfaced, so a per-request hook never spams the log. */
+	const schemaRepairNotices = new Set<string>();
 	const routeKey = (providerName: string, modelId: string): string => `${providerName}/${modelId}`;
 
+	/**
+	 * llama.cpp (and llama-swap / LM Studio / LiteLLM routes that forward to it) has
+	 * strict JSON-schema→grammar compatibility limits: root-scoped $ref resolution and,
+	 * in b10612, one exact nested maxLength parser failure. A single incompatible MCP
+	 * tool makes *every* message 400. Local endpoints get their schemas normalised;
+	 * cloud APIs stay byte-identical. See schema-repair.ts.
+	 */
+	function shouldRepairToolSchemas(provider: DiscoveredProvider, serverType: string): boolean {
+		if (provider.repairToolSchemas === false) return false;
+		if (process.env.PI_MODEL_DISCOVERY_NO_SCHEMA_REPAIR) return false;
+		if (provider.repairToolSchemas === true) return true;
+		const LOCAL_ENGINES = ["llama.cpp", "oMLX", "Ollama", "vLLM", "SGLang", "LM Studio", "llama-swap"];
+		return LOCAL_ENGINES.some((needle) => serverType.toLowerCase().includes(needle.toLowerCase())) || isLocalEndpointUrl(provider.baseUrl);
+	}
+
+	function noteToolSchemaRepair(providerName: string, report: ToolSchemaRepairReport): void {
+		if (!report.changed) return;
+		const summary = describeToolSchemaRepair(report);
+		const signature = `${providerName}::${summary}`;
+		if (schemaRepairNotices.has(signature)) return;
+		schemaRepairNotices.add(signature);
+		console.error(`[model-discovery] ${providerName}: ${summary}`);
+	}
+
 	// -----------------------------------------------------------------------
 	// Provider registration with Pi's model registry
 	// -----------------------------------------------------------------------
@@ -499,6 +331,8 @@ export default async function (pi: ExtensionAPI) {
 		if (serverType === "llama.cpp" || serverType === "oMLX" || serverType === "Ollama") {
 			if (compat.supportsDeveloperRole === undefined) compat.supportsDeveloperRole = false;
 		}
+		if (shouldRepairToolSchemas(provider, serverType)) schemaRepairProviders.add(provider.name);
+		else schemaRepairProviders.delete(provider.name);
 		if (serverType === "oMLX") {
 			// Preserve the pre-profile base-model behavior. Fixed and adaptive profile
 			// aliases supply their own complete chat-template kwargs independently.
@@ -622,9 +456,27 @@ export default async function (pi: ExtensionAPI) {
 	}
 
 	pi.on("before_provider_request", (event, ctx) => {
+		let payload: unknown = event.payload;
+		let touched = false;
+
 		const active = activeThinkingRoute(ctx);
-		if (!active) return undefined;
-		return applyThinkingProfileRoute(event.payload, active.profile, active.runtime.repetitionPenaltyKey);
+		if (active) {
+			payload = applyThinkingProfileRoute(payload, active.profile, active.runtime.repetitionPenaltyKey);
+			touched = true;
+		}
+
+		// Repair local tool schemas so llama.cpp-style grammar converters accept them.
+		const providerName = ctx.model?.provider;
+		if (providerName && schemaRepairProviders.has(providerName)) {
+			const repaired = repairRequestToolSchemas(payload);
+			if (repaired.report.changed) {
+				noteToolSchemaRepair(providerName, repaired.report);
+				payload = repaired.payload;
+				touched = true;
+			}
+		}
+
+		return touched ? payload : undefined;
 	});
 
 	const updateThinkingProfileStatus = (ctx: ExtensionContext): void => {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@maheidem/model-discovery",
-  "version": "0.6.1",
+  "version": "0.7.1",
   "type": "module",
   "description": "Interactive TUI for discovering local AI endpoints and defining named thinking/sampling profiles (llama.cpp, oMLX, Ollama, vLLM, SGLang, LM Studio).",
   "keywords": [
@@ -19,7 +19,7 @@
     "url": "https://github.com/maheidem/model-discovery/issues"
   },
   "scripts": {
-    "test": "node --experimental-strip-types --test profiles.test.ts offline.test.ts"
+    "test": "node --experimental-strip-types --test profiles.test.ts offline.test.ts enrichment.test.ts schema-repair.test.ts"
   },
   "peerDependencies": {
     "@earendil-works/pi-coding-agent": ">=0.84.0",
@@ -27,6 +27,8 @@
     "typebox": "*"
   },
   "pi": {
-    "extensions": ["index.ts"]
+    "extensions": [
+      "index.ts"
+    ]
   }
 }