@hawkeyexl/inference 0.1.0 → 0.2.0

package/README.md

@@ -1,46 +1,56 @@
 # @hawkeyexl/inference
 
 Shared LLM inference layer for the docs-as-tests toolchain: schema-constrained completion across
-Anthropic, OpenAI-compatible, and Claude CLI providers, with result caching, cost accounting, and
-an LLM-as-judge ensemble on top.
+Anthropic, OpenAI-compatible, Claude CLI, and in-process local (llama.cpp) providers, with result
+caching, cost accounting, and an LLM-as-judge ensemble on top.
 
 Extracted from three projects that had each grown their own copy —
 [docevals](https://github.com/hawkeyexl/docevals), [dockg](https://github.com/hawkeyexl/dockg), and
 [agentevals](https://github.com/hawkeyexl/agentevals) — so a provider fix lands once instead of
 three times.
 
+**📖 [Documentation](https://hawkeyexl.github.io/inference/)**
+
 ## Install
 
 ```bash
 npm install @hawkeyexl/inference
 ```
 
-Requires Node 24+. ESM only.
+Requires Node 24+. Three runtime dependencies, plus one optional peer dependency for local models.
+
+**ESM only.** The `exports` map has no `require` condition, so
+`require("@hawkeyexl/inference")` fails with `ERR_PACKAGE_PATH_NOT_EXPORTED`. From CommonJS, use
+`await import("@hawkeyexl/inference")`.
 
 ## What it does
 
-Every consumer of this library wants the same narrow thing: **send a system prompt, a user prompt,
-and a JSON Schema; get back JSON that validates against that schema, or a recorded error.** No
-streaming, no multi-turn, no tool loops.
+Every consumer wants the same narrow thing: **send a system prompt, a user prompt, and a JSON
+Schema; get back JSON that validates against that schema, or a recorded error.**
+
+```ts
+(system, user, schema, temperature) -> JSON
+```
+
+**No streaming, no multi-turn, no tool loops.** If you need a conversation, this is the wrong
+package. Widening the provider contract requires an [ADR](adrs).
 
 Two layers, one entry point:
 
-- **Completion** — the provider contract, four providers, a content-addressed cache, a price table,
-  and a validate-and-retry wrapper. This is all dockg-style structured extraction needs.
+- **Completion** — the provider contract, five providers, a content-addressed cache, a price table,
+  and a validate-and-retry wrapper. All that structured extraction needs.
 - **Judge** — the canonical verdict schema, an N-run ensemble, consensus math, and confidence-zone
   routing. Built on the completion layer; ignore it if you do not need it.
 
 ## Quick start
 
-### Schema-constrained completion
+No API key required — `MockProvider` is exported for exactly this.
 
 ```ts
-import { completeValidatedJSON, makeProvider } from "@hawkeyexl/inference";
+import { MockProvider, completeValidatedJSON } from "@hawkeyexl/inference";
 
-const provider = makeProvider({ provider: "anthropic", model: "claude-sonnet-4-5" });
-
-const run = await completeValidatedJSON<{ summary: string }>({
-  provider,
+const run = await completeValidatedJSON({
+  provider: new MockProvider([{ json: { summary: "Covers authentication." } }]),
   system: "You summarize documentation pages.",
   user: pageBody,
   schema: {
@@ -55,145 +65,48 @@ if (run.error) console.error(run.error);
 else console.log(run.result.summary, run.usage);
 ```
 
-`completeValidatedJSON` never throws on a model failure and never coerces a bad response. It
-retries once, then returns a run with `error` set and `result` absent.
+`completeValidatedJSON` never throws on a model failure and never coerces a bad response. It retries
+once, then returns a run with `error` set and `result` absent.
 
-### LLM-as-judge
+Point it at a real model by swapping the provider — or omit `provider` entirely and let the library
+detect one this machine can use, ending at the free local model:
 
 ```ts
-import { judge, makeProvider } from "@hawkeyexl/inference";
-
-const consensus = await judge({
-  provider: makeProvider({ provider: "claude-cli" }),
-  system: "You evaluate whether a page satisfies an assertion.",
-  user: "# Assertion\nThe page documents authentication.\n\n# Page\n...",
-  runs: 3,
-});
-
-consensus.verdict;   // "pass" | "fail"   — partial counts as fail
-consensus.zone;      // "auto-pass" | "auto-fail" | "human-review"
-consensus.agreement; // 0..1 across non-errored runs
+const provider = await makeProviderAsync({});
 ```
 
-Only a **unanimous, high-confidence** ensemble auto-resolves. Anything split, low-confidence, or
-containing an errored run routes to `human-review` — an errored run can never produce a silent pass.
-
-### Caching
-
-Key composition stays with you, because each consumer has a different notion of what should
-invalidate an entry (page body, prompt version, ensemble size, requested fields):
-
-```ts
-import { JsonCache, buildCacheKey, runEnsemble, sha256 } from "@hawkeyexl/inference";
-
-const cache = new JsonCache(".mytool/cache", true, "mytool");
-const cacheKey = buildCacheKey([
-  provider.provider(),
-  provider.modelName(),
-  `v${MY_PROMPT_VERSION}`,
-  `r${runs}`,
-  sha256(pageBody),         // pre-hash long parts
-]);
-
-const judgeRuns = await runEnsemble({ provider, system, user, runs, cache, cacheKey });
-```
-
-Cached runs come back flagged `cached: true`, so `costOfRuns` correctly charges nothing for a
-replay. Cache write failures warn once and continue — a read-only workspace must not abort a run
-whose inference already succeeded and was already paid for.
-
-### Cost
-
-```ts
-import { costOfRuns, pricingFor } from "@hawkeyexl/inference";
-
-const pricing = pricingFor(provider.modelName(), configOverride);
-const usd = costOfRuns(judgeRuns, pricing);
-```
-
-An unknown model returns `undefined` pricing and costs `0` — **unknown, never a guess**. A
-fabricated price is worse than an absent one when a budget gate depends on it.
-
 ## Providers
 
-Constructed through `makeProvider(spec)`. The spec is a flat, library-owned shape — map your own
-config into it rather than passing your config object (see
-[ADR 01000](adrs/01000-library-owned-provider-spec.md)).
-
-| `provider` | Structured output via | Key | Reports usage |
-|---|---|---|---|
+| `provider` | Structured output via | Credential | Reports usage |
+|---|---|---|:---:|
 | `anthropic` | forced tool call | `ANTHROPIC_API_KEY` | yes |
 | `openai` | strict `json_schema`, falls back to `json_object` | `OPENAI_API_KEY` | yes |
-| `claude-cli` | schema in the prompt, `--output-format json` | local `claude` auth | no |
+| `claude-cli` | schema in the prompt, `--output-format json` | local `claude` auth | **no** |
+| `llama-cpp` | GBNF grammar compiled from the schema | — (runs locally) | yes |
 | `mock` | scripted responses | — | synthetic |
 
-```ts
-interface ProviderSpec {
-  provider: "anthropic" | "openai" | "claude-cli" | "mock";
-  model?: string | null;      // null/undefined -> per-provider default
-  apiKeyEnv?: string | null;  // default ANTHROPIC_API_KEY / OPENAI_API_KEY
-  baseUrl?: string;           // openai only, default https://api.openai.com/v1
-  command?: string;           // claude-cli only, default "claude"
-  timeoutMs?: number;         // claude-cli only, default 180000
-  pricing?: Pricing;          // override the built-in table
-  anthropic?: AnthropicProviderOptions;   // e.g. toolName, maxTokens
-  openai?: OpenAICompatProviderOptions;   // e.g. schemaName
-  exec?: ExecFn;              // test seam for claude-cli
-  mockResponses?: MockResponse[];
-}
-```
-
-`resolveProviderIdentity(spec)` returns `{ provider, model }` **without constructing anything** —
-cache keys and pricing need the identity, but a fully-cached run should not require an API key.
-
-Notes on the non-obvious bits:
-
-- **`openai`** targets any `/chat/completions` server (OpenAI, Azure, Ollama, Groq, Together). It
-  prefers strict `json_schema`, and `toStrictSchema` rewrites your schema into the strict subset
-  (every property in `required`, optionality as a `null` type union, unsupported keywords dropped);
-  nulls are stripped back out of the response. If the server rejects `response_format`, it
-  permanently falls back to `json_object` with the schema in the prompt. Keyless local servers are
-  allowed — only `api.openai.com` requires a key.
-- **`claude-cli`** uses your local Claude CLI auth, so no API key. The prompt goes over **stdin**,
-  never argv: user content routinely exceeds the ~32K Windows command-line limit.
-
-## Testing against this library
-
-`MockProvider` is exported for exactly this. No network required:
-
-```ts
-import { MockProvider, mockVerdict, runEnsemble } from "@hawkeyexl/inference";
-
-const provider = new MockProvider([mockVerdict("pass", 0.95)]);   // cycles when exhausted
-const runs = await runEnsemble({ provider, system, user, runs: 3 });
-provider.requests;  // every request seen, in order
-```
-
-Script an error with `{ error: "429 rate limited" }` to exercise your failure paths.
-
-## API
-
-Everything exports from the package root.
-
-**Providers** — `makeProvider`, `resolveProviderIdentity`, `DEFAULT_MODELS`,
-`DEFAULT_OPENAI_BASE_URL`, `AnthropicProvider`, `OpenAICompatProvider`, `ClaudeCliProvider`,
-`MockProvider`, `mockVerdict`, `extractJson`, `toStrictSchema`, `stripNulls`, `realExec`
-
-**Completion** — `completeValidatedJSON`, `validatorFor`
-
-**Cache** — `JsonCache`, `buildCacheKey`, `sha256`
+Omit `provider` and the highest-priority one this machine can actually use is detected, ending at
+`llama-cpp` — which needs no key, and whose native binding is installed on demand into
+`~/.hawkeyexl-inference/runtime` if it is missing. That install warns once and is refused by
+`INFERENCE_NO_AUTO_INSTALL`; it never touches your `package.json`, lockfile or `node_modules`.
 
-**Cost** — `pricingFor`, `costOfUsage`, `costOfRuns`, `PRICE_TABLE`
+Usage reporting is the column that decides whether cost accounting works: a provider that reports no
+tokens makes a budget gate inert. See
+[Choose a provider](https://hawkeyexl.github.io/inference/get-started/choose-a-provider/).
 
-**Judge** — `judge`, `runEnsemble`, `computeConsensus`, `zoneFor`, `VERDICT_SCHEMA`,
-`DEFAULT_ZONES`
+## Documentation
 
-**Errors** — `InferenceError` (operational failures: missing key, unknown provider)
+| Track | What it covers |
+|---|---|
+| [Get started](https://hawkeyexl.github.io/inference/get-started/) | Install, one validated call with no key, choosing a provider |
+| [Judge & consensus](https://hawkeyexl.github.io/inference/judge/) | Ensembles, consensus math, confidence zones, caching, budgets |
+| [Structured extraction](https://hawkeyexl.github.io/inference/extract/) | One schema-constrained call, honest failures, the subprocess seam |
+| [Run models locally](https://hawkeyexl.github.io/inference/local/) | GGUF weights in-process, model selection, managing weights on disk |
+| [Keep it working](https://hawkeyexl.github.io/inference/keep-it-working/testing/) | Testing without a network, upgrading without losing a cache |
+| [Reference](https://hawkeyexl.github.io/inference/reference/providers/) | Full signatures for every export |
 
-Types: `InferenceProvider`, `ProviderSpec`, `ProviderName`, `CompleteJSONRequest`,
-`CompleteJSONResponse`, `InferenceRun`, `TokenUsage`, `Pricing`, `JudgeRun`, `JudgeVerdict`,
-`ConsensusResult`, `Match`, `Zone`, `ZoneThresholds`, `EnsembleOptions`, `ExecFn`, `ExecResult`,
-`ExecOptions`, `MockResponse`.
+Who the docs serve and why each page exists lives in
+[docs/content-strategy/](docs/content-strategy).
 
 ## Design decisions
 
@@ -205,6 +118,19 @@ Recorded as ADRs in [adrs/](adrs):
   canonical verdict schema with a per-consumer override seam
 - [01002](adrs/01002-best-of-merge-of-three-forks.md) — which fork won for each merged file, so the
   losing variants are not reintroduced
+- [01003](adrs/01003-in-process-local-models-via-node-llama-cpp.md) — in-process local models via
+  node-llama-cpp, why selectors need an async factory, and why the catalog pins exact blob paths
+- [01004](adrs/01004-provider-auto-detection.md) — detect an available provider when none is
+  specified, ending at the local model
+- [01005](adrs/01005-docset-strategy-and-executable-examples.md) — a CUJ-first documentation set,
+  with samples that CI executes
+- [01006](adrs/01006-documenting-failure-and-orchestration.md) — document failure and orchestration,
+  and gate both against the source
+- [01007](adrs/01007-harden-two-operational-failure-paths.md) — harden two operational failure paths:
+  non-JSON CLI output, and an unsupported Node
+- [01008](adrs/01008-auto-install-the-local-runtime.md) — auto-install the local runtime into a
+  library-owned prefix, why the shim beats `createRequire`, and why a model without a provider is
+  now an error
 
 ## License
 

package/dist/index.d.ts

@@ -9,6 +9,9 @@ declare class InferenceError extends Error {
     constructor(message: string);
 }
 
+/** Test seam: reset the once-per-process Node version warning. */
+declare function resetNodeVersionWarning(): void;
+
 /**
  * The provider contract. A provider turns a (system, user, schema) request
  * into schema-conforming JSON. `provider()` and `modelName()` feed cache keys
@@ -173,9 +176,110 @@ declare function mockVerdict(match: "pass" | "fail" | "partial", confidence: num
     json: unknown;
 };
 
-type ProviderName = "anthropic" | "openai" | "claude-cli" | "mock";
+interface LlamaPromptOptions {
+    /** JSON Schema converted to a GBNF grammar by the runtime. */
+    schema: Record<string, unknown>;
+    temperature: number;
+    /** Thinking budget; 0 disables it. See the note in `completeJSON`. */
+    thoughtTokens: number;
+    maxTokens?: number;
+}
+interface LlamaPromptResult {
+    text: string;
+    usage?: TokenUsage;
+    /**
+     * Why generation stopped. `"maxTokens"` means the output was cut off, so the
+     * text is almost certainly truncated JSON — see the guard in `completeJSON`.
+     */
+    stopReason?: string;
+}
+interface LlamaSession {
+    prompt(text: string, options: LlamaPromptOptions): Promise<LlamaPromptResult>;
+    dispose(): Promise<void>;
+}
+interface LlamaLoadedModel {
+    createSession(systemPrompt: string): Promise<LlamaSession>;
+    dispose(): Promise<void>;
+}
+/**
+ * The whole of `node-llama-cpp` that this provider uses. Kept this narrow so a
+ * test fake is a few lines and so the real adapter is the only place that
+ * knows the upstream API shape.
+ */
+interface LlamaRuntime {
+    /**
+     * Resolve an `hf:` URI or path to a local file inside `directory`,
+     * downloading if needed.
+     */
+    resolveModelFile(uri: string, directory: string): Promise<string>;
+    loadModel(path: string): Promise<LlamaLoadedModel>;
+    /** Memory available for weights, in bytes — VRAM if there is a GPU, else RAM. */
+    getMemoryBudgetBytes(): Promise<number>;
+}
+interface LlamaCppProviderOptions {
+    /** Injected for tests; defaults to the real `node-llama-cpp` adapter. */
+    runtime?: LlamaRuntime;
+    /**
+     * Thinking budget in tokens, default 0.
+     *
+     * Gemma 4 has a thinking mode, but a grammar constrains generation from
+     * token 0 — so an unbudgeted model starts reasoning and gets cut off
+     * mid-thought. Zero is the deterministic choice for judging; raise it if you
+     * want reasoning before the JSON.
+     */
+    thoughtTokens?: number;
+    maxTokens?: number;
+    /**
+     * Where to download and look for weights. Defaults to this library's own
+     * directory — see `defaultLlamaModelsDirectory`.
+     */
+    modelsDirectory?: string;
+}
+/**
+ * Free every loaded model.
+ *
+ * A standalone function rather than a `dispose()` on `InferenceProvider`:
+ * adding one to the contract would make all five providers carry a lifecycle
+ * only this one has. Short-lived processes can skip it.
+ */
+declare function disposeLlamaModels(): Promise<void>;
+declare class LlamaCppProvider implements InferenceProvider {
+    private readonly model;
+    private readonly uri;
+    private readonly runtime;
+    private readonly thoughtTokens;
+    private readonly maxTokens;
+    private readonly modelsDirectory;
+    /**
+     * Loaded-model key: the same URI in two directories is two different files.
+     * Built with `buildCacheKey` so its parts are length-prefixed — a plain join
+     * would let two different (directory, uri) pairs collide and hand a provider
+     * back the wrong weights.
+     */
+    private readonly cacheKey;
+    constructor(model: string, options?: LlamaCppProviderOptions);
+    provider(): string;
+    modelName(): string;
+    completeJSON(req: CompleteJSONRequest): Promise<CompleteJSONResponse>;
+    private load;
+}
+/**
+ * Lazy adapter over the real `node-llama-cpp`. Every method defers to the
+ * dynamic import, so constructing a provider for a fully-cached run never
+ * loads the native binary.
+ */
+declare function defaultLlamaRuntime(): LlamaRuntime;
+
+type ProviderName = "anthropic" | "openai" | "claude-cli" | "mock" | "llama-cpp";
+/** A concrete provider, or `"auto"` to detect one. */
+type ProviderSelector = ProviderName | "auto";
 interface ProviderSpec {
-    provider: ProviderName;
+    /**
+     * Omitting this is identical to `"auto"`: the highest-priority provider this
+     * machine can actually use is detected, ending at the free local model.
+     * Resolving it needs `makeProviderAsync`/`resolveProviderIdentityAsync`.
+     */
+    provider?: ProviderSelector;
     /** null/undefined selects the per-provider default. */
     model?: string | null;
     /** Env var NAME holding the API key; null/undefined selects the default. */
@@ -195,23 +299,77 @@ interface ProviderSpec {
     /** Provider-specific tuning, ignored by the other providers. */
     anthropic?: AnthropicProviderOptions;
     openai?: OpenAICompatProviderOptions;
+    llamaCpp?: LlamaCppProviderOptions;
     /** Test seam for the claude-cli provider. */
     exec?: ExecFn;
+    /** Test seam for the llama-cpp provider. */
+    llamaRuntime?: LlamaRuntime;
     /** Scripted responses for the mock provider; defaults to a single empty object. */
     mockResponses?: MockResponse[];
 }
 declare const DEFAULT_MODELS: Record<ProviderName, string>;
 declare const DEFAULT_OPENAI_BASE_URL = "https://api.openai.com/v1";
+interface ProviderIdentity {
+    /** Always concrete — never `"auto"`, so it is safe as cache-key material. */
+    provider: ProviderName;
+    model: string;
+}
 /**
  * Resolve the provider name and model WITHOUT constructing the provider —
  * cache keys and pricing need the identity, but construction may require an
  * API key that a fully-cached run never uses.
+ *
+ * Throws for an unresolved `llama-cpp` selector. That is deliberate: picking a
+ * tier weighs GPU VRAM, which needs an await, and returning the literal
+ * "auto" as cache-key material would let a 2 GB and a 12 GB model share cached
+ * verdicts — and give different results per machine under one key. Use
+ * `resolveProviderIdentityAsync` for selectors.
  */
-declare function resolveProviderIdentity(spec: ProviderSpec): {
-    provider: string;
-    model: string;
-};
+declare function resolveProviderIdentity(spec: ProviderSpec): ProviderIdentity;
+/**
+ * Selector-aware identity resolution. Returns the CONCRETE model a selector
+ * resolved to, so the cache key names the weights that actually ran.
+ *
+ * Every other provider delegates to the synchronous form, so a consumer can
+ * switch to this wholesale.
+ */
+declare function resolveProviderIdentityAsync(spec: ProviderSpec): Promise<ProviderIdentity>;
 declare function makeProvider(spec: ProviderSpec): InferenceProvider;
+/**
+ * Selector-aware provider construction. Resolves a `llama-cpp` selector
+ * against this machine first, so the returned provider's `modelName()` — and
+ * therefore the cache key — names the weights it will actually load.
+ *
+ * Every other provider delegates to `makeProvider`.
+ */
+declare function makeProviderAsync(spec: ProviderSpec): Promise<InferenceProvider>;
+
+/**
+ * Priority order. `mock` is deliberately absent: it answers `{ json: {} }`
+ * unless scripted, which would sail through as a non-error result — the exact
+ * opposite of the "an errored run is recorded, never coerced" invariant the
+ * consuming eval tools depend on. It must always be asked for by name.
+ */
+declare const DETECTION_ORDER: readonly ProviderName[];
+/** Test seam: forget the memoised Claude CLI probes. */
+declare function resetClaudeCliProbe(): void;
+/**
+ * Every provider this machine could use right now, in priority order.
+ *
+ * Useful for showing a picker or explaining a fallback; `detectProvider` is
+ * the same sweep with the first hit returned.
+ */
+declare function availableProviders(spec?: ProviderSpec): Promise<ProviderName[]>;
+/**
+ * The highest-priority provider this machine can use.
+ *
+ * Throws an `InferenceError` naming every provider and why each was
+ * unavailable — far more actionable than the `Unknown provider "undefined"`
+ * this replaces.
+ */
+declare function detectProvider(spec?: ProviderSpec): Promise<ProviderName>;
+/** Test seam: reset the once-per-process auto-detection warnings. */
+declare function resetProviderDetectionWarning(): void;
 
 declare class ClaudeCliProvider implements InferenceProvider {
     private readonly model;
@@ -224,6 +382,179 @@ declare class ClaudeCliProvider implements InferenceProvider {
     completeJSON(req: CompleteJSONRequest): Promise<CompleteJSONResponse>;
 }
 
+/**
+ * Where this library downloads weights — its OWN directory, not
+ * node-llama-cpp's global `~/.node-llama-cpp/models`.
+ *
+ * That default is shared: node-llama-cpp's CLI writes there, as does anything
+ * else on the machine using it. Owning a directory outright means clearing it
+ * can never destroy a model this library did not download, and it keeps one
+ * copy shared across every consumer of this package on the machine.
+ *
+ * `INFERENCE_MODELS_DIR` overrides it — useful for CI or a shared volume.
+ */
+declare function defaultLlamaModelsDirectory(): string;
+/** Size tiers, smallest first. Order is load-bearing for `tierForBudget`. */
+declare const LLAMA_TIERS: readonly ["fast", "balanced", "quality"];
+type LlamaTier = (typeof LLAMA_TIERS)[number];
+/** Model selectors — resolved against hardware, never used as a cache key. */
+declare const LLAMA_SELECTORS: readonly ["auto", "fast", "balanced", "quality"];
+type LlamaSelector = (typeof LLAMA_SELECTORS)[number];
+interface LlamaModelEntry {
+    /** `hf:` URI pinned to one blob, handed to `resolveModelFile` as-is. */
+    readonly uri: string;
+    /** Size of that blob in bytes, as reported by the Hugging Face API. */
+    readonly sizeBytes: number;
+    readonly license: string;
+    /** Absent for entries that are selectable by alias but not by tier. */
+    readonly tier?: LlamaTier;
+    /** Human note for `LLAMA_MODELS` readers deciding what to download. */
+    readonly notes: string;
+}
+/**
+ * Frozen per entry, not just at the top level.
+ *
+ * A shallow freeze leaves the entries writable, and this catalog is exported
+ * for consumers to read: a stray write to `sizeBytes` silently re-points
+ * `tierForBudget` process-wide, and a write to `uri` defeats the pinned-blob
+ * invariant the whole catalog exists to hold (ADR 01003).
+ */
+declare const LLAMA_MODELS: Readonly<Record<string, LlamaModelEntry>>;
+declare function isLlamaSelector(model: string): model is LlamaSelector;
+/**
+ * Largest tier whose weights fit the memory budget with headroom. Lands at
+ * roughly: >=24 GB -> quality, >=15 GB -> balanced, else fast.
+ *
+ * Sized off the catalog's recorded bytes rather than parameter counts: Gemma
+ * 4's E-series are per-layer-embedding models whose footprint does not track
+ * "effective params" (E4B is 4.5B effective but 15 GB at BF16).
+ */
+declare function tierForBudget(budgetBytes: number): LlamaTier;
+/**
+ * Catalog alias backing a tier. Selectors resolve to this rather than to a raw
+ * URI so the identity — and therefore the cache key — stays human-readable.
+ */
+declare function aliasForTier(tier: LlamaTier): string;
+/** The pinned URI backing a tier keyword. */
+declare function uriForTier(tier: LlamaTier): string;
+/**
+ * Turn a concrete model reference — a curated alias, an `hf:` URI, or a local
+ * path — into something `resolveModelFile` accepts.
+ *
+ * Selectors are rejected rather than guessed at: they need a hardware probe,
+ * which is async, and this runs on the synchronous cache-key path. Resolving
+ * one here from RAM alone would emit a key naming a model the provider then
+ * did not load.
+ */
+declare function resolveLlamaModelRef(model: string): string;
+/**
+ * The catalog's pinned filename for a model reference.
+ *
+ * Strips a `#branch` fragment (node-llama-cpp accepts
+ * `hf:user/repo/file.gguf#branch`) and splits on both separators, so a Windows
+ * path resolves too. Getting either wrong makes callers silently match nothing.
+ */
+declare function blobNameFor(model: string): string;
+/**
+ * Are this model's weights already on disk?
+ *
+ * A `.ipull` partial counts as NOT downloaded: it cannot be loaded, so
+ * treating it as present would skip the download warning and then stall on a
+ * download anyway.
+ */
+declare function isModelDownloaded(model: string, directory: string): boolean;
+
+interface ClearedModelFile {
+    path: string;
+    sizeBytes: number;
+}
+interface ClearLlamaModelsResult {
+    /** Files removed, or that would be removed under `dryRun`. */
+    files: ClearedModelFile[];
+    freedBytes: number;
+    directory: string;
+    dryRun: boolean;
+}
+interface ClearLlamaModelsOptions {
+    /** Defaults to this library's own models directory. */
+    directory?: string;
+    /** Clear only these models (alias or `hf:` URI); default clears all. */
+    models?: string[];
+    /** Report what would be removed without deleting anything. */
+    dryRun?: boolean;
+}
+/**
+ * Delete downloaded GGUF weights and interrupted partial downloads.
+ *
+ * Loaded models are disposed first: on Windows the weights are memory-mapped
+ * while loaded, and deleting an open file fails with EBUSY/EPERM.
+ */
+declare function clearLlamaModels(options?: ClearLlamaModelsOptions): Promise<ClearLlamaModelsResult>;
+
+interface RuntimeInstallOptions {
+    /** Defaults to this library's own runtime directory. */
+    directory?: string;
+    /** Injected for tests; defaults to the real process runner. */
+    exec?: ExecFn;
+    /** Injected for tests; defaults to `process.env`. */
+    env?: Record<string, string | undefined>;
+    timeoutMs?: number;
+    /** Test seam: how the shim is imported once it exists. */
+    importShim?: (url: string) => Promise<unknown>;
+    /**
+     * Test seam: how the consumer's own copy is looked for.
+     *
+     * Whether an optional peer is installed is a property of the machine, and the
+     * behaviour that matters most here — that probing installs nothing — is
+     * unobservable on a machine that already has it. Injecting the lookup is the
+     * only way to assert it deterministically.
+     */
+    probeImport?: () => Promise<unknown>;
+}
+/**
+ * Where this library installs the binding — its OWN directory, beside the
+ * models directory and for the same reason.
+ *
+ * `INFERENCE_RUNTIME_DIR` overrides it, mirroring `INFERENCE_MODELS_DIR`.
+ */
+declare function defaultLlamaRuntimeDirectory(env?: Record<string, string | undefined>): string;
+/** Whether the binding can be used, and at what cost, WITHOUT installing it. */
+type RuntimeStatus = 
+/** Importable right now — either the consumer's own copy or a filled prefix. */
+{
+    state: "present";
+}
+/** Absent, but an install is permitted. Using it will fetch a native module. */
+ | {
+    state: "installable";
+    directory: string;
+}
+/** Absent and installing is refused, so this provider cannot be used. */
+ | {
+    state: "refused";
+    reason: string;
+};
+/**
+ * Can the local runtime be used, and would using it install anything?
+ *
+ * Detection needs this rather than simply calling `getMemoryBudgetBytes`:
+ * that goes through `importNodeLlamaCpp` and would install, which turns
+ * `availableProviders()` — a function whose entire job is to REPORT what is
+ * usable — into one that changes what is usable. Probing must not have the side
+ * effect it is probing for.
+ */
+declare function nodeLlamaCppStatus(options?: RuntimeInstallOptions): Promise<RuntimeStatus>;
+/** Test seam: forget in-flight installs and re-arm the one-time warning. */
+declare function resetRuntimeInstall(): void;
+/**
+ * Import `node-llama-cpp` from the library-owned prefix, installing it first if
+ * it is not there.
+ *
+ * Callers reach this only after a plain `import("node-llama-cpp")` has already
+ * failed — a consumer who installed the peer themselves never gets here.
+ */
+declare function importNodeLlamaCpp(options?: RuntimeInstallOptions): Promise<unknown>;
+
 declare const realExec: ExecFn;
 
 /** One attempt at a schema-constrained completion. */
@@ -375,4 +706,4 @@ declare function judge(options: EnsembleOptions & {
 /** Test seam: reset the once-per-process temperature warning. */
 declare function resetTemperatureWarning(): void;
 
-export { AnthropicProvider, type AnthropicProviderOptions, ClaudeCliProvider, type CompleteJSONRequest, type CompleteJSONResponse, type CompleteValidatedOptions, type ConsensusResult, DEFAULT_MODELS, DEFAULT_OPENAI_BASE_URL, DEFAULT_ZONES, type EnsembleOptions, type ExecFn, type ExecOptions, type ExecResult, InferenceError, type InferenceProvider, type InferenceRun, JsonCache, type JudgeRun, type JudgeVerdict, type Match, MockProvider, type MockResponse, OpenAICompatProvider, type OpenAICompatProviderOptions, PRICE_TABLE, type Pricing, type ProviderName, type ProviderSpec, type TokenUsage, VERDICT_SCHEMA, type Zone, type ZoneThresholds, buildCacheKey, completeValidatedJSON, computeConsensus, costOfRuns, costOfUsage, extractJson, judge, makeProvider, mockVerdict, pricingFor, realExec, resetTemperatureWarning, resolveProviderIdentity, runEnsemble, sha256, stripNulls, toStrictSchema, validatorFor, zoneFor };
+export { AnthropicProvider, type AnthropicProviderOptions, ClaudeCliProvider, type ClearLlamaModelsOptions, type ClearLlamaModelsResult, type ClearedModelFile, type CompleteJSONRequest, type CompleteJSONResponse, type CompleteValidatedOptions, type ConsensusResult, DEFAULT_MODELS, DEFAULT_OPENAI_BASE_URL, DEFAULT_ZONES, DETECTION_ORDER, type EnsembleOptions, type ExecFn, type ExecOptions, type ExecResult, InferenceError, type InferenceProvider, type InferenceRun, JsonCache, type JudgeRun, type JudgeVerdict, LLAMA_MODELS, LLAMA_SELECTORS, LLAMA_TIERS, LlamaCppProvider, type LlamaCppProviderOptions, type LlamaLoadedModel, type LlamaModelEntry, type LlamaPromptOptions, type LlamaPromptResult, type LlamaRuntime, type LlamaSelector, type LlamaSession, type LlamaTier, type Match, MockProvider, type MockResponse, OpenAICompatProvider, type OpenAICompatProviderOptions, PRICE_TABLE, type Pricing, type ProviderIdentity, type ProviderName, type ProviderSelector, type ProviderSpec, type RuntimeInstallOptions, type RuntimeStatus, type TokenUsage, VERDICT_SCHEMA, type Zone, type ZoneThresholds, aliasForTier, availableProviders, blobNameFor, buildCacheKey, clearLlamaModels, completeValidatedJSON, computeConsensus, costOfRuns, costOfUsage, defaultLlamaModelsDirectory, defaultLlamaRuntime, defaultLlamaRuntimeDirectory, detectProvider, disposeLlamaModels, extractJson, importNodeLlamaCpp, isLlamaSelector, isModelDownloaded, judge, makeProvider, makeProviderAsync, mockVerdict, nodeLlamaCppStatus, pricingFor, realExec, resetClaudeCliProbe, resetNodeVersionWarning, resetProviderDetectionWarning, resetRuntimeInstall, resetTemperatureWarning, resolveLlamaModelRef, resolveProviderIdentity, resolveProviderIdentityAsync, runEnsemble, sha256, stripNulls, tierForBudget, toStrictSchema, uriForTier, validatorFor, zoneFor };