@happyvertical/smrt-inference 0.0.0-stage → 0.60.16

package/AGENTS.md

@@ -0,0 +1,206 @@
+# smrt-inference
+
+Binding behavior for core's inference tiers (epic #4749). Core owns the names
+and the seam ([core inference.md](../core/agents/inference.md)); this package
+owns alias chains, classified fallback, ladder degradation and
+`smrt doctor --ai` (#4752), the fusion interpreter (#4753), the settings
+panel with layered binding storage (#4754), per-user and per-tenant
+bindings (#4756), candidate capability checks (#4758), and hosted
+provisioning and the spend gate (#4759, #4760). App-author guide:
+[docs/content/inference.md](../../docs/content/inference.md).
+
+## Entries
+
+- Root (browser-safe): importing it calls `registerInference()`, which sets
+  the executor (`alias`, `candidates`, `fusion`) and a validating resolver
+  (`setInferenceResolver()`). The resolver binds nothing: it validates chain
+  config, then defers to a resolver registered earlier (#4756 per-user
+  bindings, `./resolver`) or to core's built-in config resolution. No aliases and no chain
+  targets means no change in behavior. `browser-safe.test.ts` holds the graph
+  free of `node:`, the AI SDK and smrt-config by walking its import graph;
+  the bundle-gate browser fixture also builds it (the package has a model).
+- `./node`: `checkInferenceBindings()`, `probeInferenceCandidate()`,
+  `inferenceProbeUrl()`, `formatInferenceDoctorReport()`; the CLI's
+  `doctor --ai` and #4754's test connection call these. Also hosted
+  provisioning (`src/hosted.ts`, #4759).
+- `./resolver` (Node-only, #4756): `createUserInferenceResolver()` and
+  `registerUserInferenceResolver()` over smrt-secrets / tenancy / users. The
+  root re-exports only the browser-safe stored-config shape
+  (`src/provider-config.ts`): parse, redact, tier → target map.
+- `./models` (browser-safe, held by `browser-safe.test.ts` and the
+  bundle-gate browser fixture): the `_smrt_inference_overrides` model;
+  generated consumer registration imports it from here.
+- `./server` (Node-only): settings service, override service, override
+  resolver, key rule, preference kind (#4754). `./svelte`: the panel.
+- `./gate` (Node-only, #4760): the hosted spend gate and kill switch
+  (below). `browser-safe.test.ts` keeps it off the root graph.
+
+## Per-user and per-tenant bindings (#4756)
+
+`createUserInferenceResolver()` resolves per tier: the user's stored config,
+the tenant's, `fallback`, the app `inference` block, then the configured
+client; every resolution carries `target.source` (`byo | local | hosted |
+app`, the `source` usage tag). Keys live only in the smrt-secrets type
+`inference.provider-config` (`context = tenantId`), every stored binding is
+self-contained (own `baseUrl` + `apiKey`, or a preset), the user layer
+applies only to a principal of that tenant, and agents pass `userId: null`
+(tenant layer only). The stored config also carries `mode` (#4757).
+Details: [agents/resolver.md](agents/resolver.md).
+
+## Resolver composition
+
+One resolver is registered, outermost first: the #4752 validating wrapper,
+the #4756 user resolver, its `fallback`, then core's config resolution.
+Hosts using the #4754 override table pass
+`createInferenceOverrideResolver({ db })` as that `fallback`:
+`registerUserInferenceResolver(createUserInferenceResolver({ db, fallback:
+createInferenceOverrideResolver({ db }) }))`. Order per tier: user stored
+binding, tenant stored binding, allowed tenant override row, app override
+row, `inference` config, configured client. Keys are stored once, in
+`inference.provider-config`: the host turns #4754's origin-bound
+`bindInferenceKey()` record into a self-contained binding with
+`inferenceProviderConfigFromKey(record, { model })` and writes it through
+the user resolver's `write(level, config)`; never a second secret type.
+
+## Hosted inference (#4759, #4760)
+
+`./node` provisions per-user gateway keys through `ai.admin` of the
+`bifrost` / `litellm` provider (`SMRT_INFERENCE_HOSTED_*`, read once), storing
+the key only in the user's `inference.provider-config` (`source: 'hosted'`).
+Plan changes update the key budget, revoke ends local use and then deletes the
+key (failures are owed and retried by reconcile), and the tenant project is
+looked up before one is created (`@happyvertical/ai` >= 0.111.7). `./gate`'s
+`registerHostedInferenceGate({ killSwitch, spend })` is core's
+`InferenceGate`: kill switch `inference.hosted.enabled` (a global DISABLE
+always wins), tenant, entitlement, then plan spend caps; a block throws
+`InferenceUnavailableError` (`entitlement`) and writes a `blocked` usage row.
+Details: [agents/hosted.md](agents/hosted.md).
+
+## Config
+
+```ts
+config({ inference: {
+  aliases: { frontier: { candidates: [
+    { provider: 'anthropic', model: 'claude-…' },               // provider key var
+    { provider: 'openai', model: 'gpt-…', baseUrl, apiKeyEnv }, // own key + endpoint
+    { preset: 'lemonade', model: 'qwen…' },                     // #4674 preset
+    { alias: 'local' },                                         // expanded in place
+  ] } },
+  tiers: { great: { alias: 'frontier' }, cheap: { candidates: [ … ] } },
+  degrade: true, // default
+} });
+```
+
+A candidate is exactly one of provider/model, preset/model, or alias.
+`apiKeyEnv` is read at call time and needs a `baseUrl` (core's pair rule: a
+key only travels with its own endpoint); a named variable that is unset skips
+the candidate as `credential`. `findInferenceConfigIssues()` /
+`validateInferenceConfig()` name every offender by path: empty alias, unknown
+alias (tier or nested), cycle, key without endpoint, unknown preset. The
+executor also validates every resolved target it runs (a resolver's chains
+never pass through config), and a candidate key is never dropped: without
+its own `baseUrl` core's pair rule refuses it.
+
+## Executor (`src/executor.ts`)
+
+- Free of SDK imports: it drives the clients core builds through
+  `context.clientFor(target)`; tests use fake clients.
+- Failure classes (`classify.ts`): `quota | credential | missing-model |
+  transient | refusal | missing-capability`, from error class names, `code`, `retryable` and HTTP
+  status across `cause` / `error` / `context.originalError`, never message
+  text. The SDK maps 5xx and other HTTP failures to `API_ERROR` /
+  `UNKNOWN_ERROR` without a status or cause (happyvertical/sdk#1493); those
+  count as `transient`. A definitive client error that survives in the chain
+  (other 4xx, `invalid_request_error`, context length) is not a routing
+  failure. Core's typed bind refusal (`INFERENCE_TARGET_UNBOUND`: pair rule,
+  `HAVE_AI_*`) is `credential` and advances. Aborts and unclassified errors stop the walk and are rethrown (an
+  unclassified one is still recorded).
+- `transient` retries the same candidate once after 200–500 ms of jitter;
+  every class then advances. A `refusal` finish reason with nothing streamed
+  also advances; finish callbacks are replayed only for the kept answer.
+- Streamed text is never extended: a candidate that emitted text and then
+  failed ends the walk with a `partial` substitution and
+  `InferenceExhaustedError` (`reason: 'partial'`, `partialText`), unless
+  `allowPartial` returns the text. A hosted-gate block (`AI_UNAVAILABLE`
+  from `clientFor()`, #4760) stops the walk and records nothing (core wrote
+  the `blocked` row). A step whose target names a `source` (a ladder step's
+  resolved binding) records that source over the chain's, so hosted spend
+  meters as hosted; `expandTarget()` hands a chain's own `source` to its
+  candidate steps, so a hosted chain reached on a ladder step is gated in
+  `clientFor()` and meters as hosted too. A `stream()` consumer that stops reading
+  aborts the provider stream (internal signal merged with the caller's).
+- Ladder: when a tier's chain is spent and `degrade !== false`, resolve the
+  next lower tier through `context.resolveTier()`, stopping below
+  `minimumTier`. Never up. Steps already tried (same identity) are skipped,
+  so unbound lower tiers try the configured client once.
+- Tags on the answering call: caller tags, `tierRequested`, `tierResolved`
+  (the tier that answered), and `substitutions` (JSON array of
+  `{ from, to, reason }`, labels `provider/model`, `preset:<id>/model`,
+  `default`). A call that never answered records one zero-token row through
+  `context.recordUsage()` (core's usage handlers; no second writer) with
+  `outcome: failed | partial`, `reason` and the full trail ending `to: null` (reason `unclassified` when a non-routing error ended it),
+  then throws `InferenceExhaustedError` (`INFERENCE_EXHAUSTED`).
+- Capabilities (#4758): before calling a candidate, a call that sends
+  `responseFormat` (JSON) or `tools` checks `context.capabilitiesFor(target)`
+  (core's detection, [inference.md](../core/agents/inference.md)). A
+  mismatch the caller's `capabilityFallback` option permits (`json: 'text'`,
+  `tools: 'drop'`) runs the candidate without the feature, tagged
+  `capabilityFallback`; any other is a `missing-capability` substitution and
+  the walk advances with no model call. The option never reaches a provider.
+  Core's `InferenceUnavailableError` (`reason: 'capability'`) also classifies
+  as `missing-capability`. No detector (older core) or no requirement: no
+  check. Context limits are core's (single targets only).
+- Add a class to `INFERENCE_FAILURE_CLASSES` and a kind to `EXECUTOR_KINDS`
+  (`createChainExecutor()` in `fusion.ts` dispatches on it); consumers
+  ignore kinds they do not run.
+
+## Fusions (#4753)
+
+`{ fusion: '<key>' }` tier targets run consumer-defined fusions (`single` 1
+call, `pair` 3, `handoff` 2) from, highest first, the resolver's stored app
+rows (core's `lookupFusion()`, #4800), `inference.fusions`, or
+`defineFusion()`; every source is re-validated before it runs. SMRT ships the interpreter, never a fusion. Seats are
+smrt-prompts keys run through `createChainClient()`; a dead seat never
+aborts a `pair`, all seats dead throws `InferenceFusionError`; each run writes
+one `_smrt_ai_runs` row. Fusions are defined at app scope only; a tenant
+may bind a tier to an allow-listed fusion, never define one. Details:
+[agents/fusions.md](agents/fusions.md).
+
+## Doctor (`src/node.ts`)
+
+Per tier, alias and fusion (structure only): provider recognized (AI SDK provider list or preset id),
+key variable present (a provider's default variable only warns: a
+same-provider target inherits the configured key), model named (warns),
+alias resolvable, cycle-free. `probe` GETs `/models` (OpenAI-compatible,
+Anthropic, Gemini) or Ollama `/api/tags` with a 3 s timeout, sending the key
+only to its own endpoint. A binding errors when its config is invalid or no
+candidate can answer; one bad candidate in a working chain warns. Keys never
+appear in the report. With `probe` each candidate also reports its
+capabilities and their source (#4758, `formatInferenceCapabilities()`): the
+listing's context length, `supported_parameters` and input modalities
+(`inferenceCapabilitiesFromModelListing()`) are the `probe` layer; the SDK
+`provider` layer only through the `sdkCapabilities` option (the doctor builds
+no SDK clients).
+
+## Settings UI and binding storage (#4754)
+
+`./svelte` `InferenceSettingsPanel`, `./server` `InferenceSettingsService` /
+`InferenceOverrideService` / `createInferenceOverrideResolver()`, `./models`
+the `_smrt_inference_overrides` model (ships through `smrt db:migrate`).
+Keys go to smrt-secrets through the host and are bound to their endpoint's
+origin; user choices are the smrt-preferences kind `inference`; app/tenant
+bindings are override rows, tenants limited to the app row's allow-list and
+the `tenantTargetPolicy` hook. The app's transcript setting (#4781) is the
+app-only `transcripts` row, editable in the panel only with `appEditable`,
+applied as core's transcript app settings layer; the Usage tab shows
+transcript counts and links to a host route, never content. Details: [agents/settings.md](agents/settings.md),
+consumer guide [docs/content/inference-settings.md](../../docs/content/inference-settings.md).
+
+## Validation
+
+```bash
+pnpm --filter @happyvertical/smrt-inference test
+pnpm --filter @happyvertical/smrt-inference test:postgres   # hosted provisioning on PostgreSQL (advisory lock, runOnce)
+pnpm --filter @happyvertical/smrt-inference typecheck
+pnpm --filter @happyvertical/smrt-inference build
+```

package/CLAUDE.md

@@ -0,0 +1 @@
+@AGENTS.md

package/LICENSE

@@ -0,0 +1,7 @@
+Copyright <2025> <Happy Vertical Corporation>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

package/README.md

@@ -1,3 +1,74 @@
-# Temporary Holding Version
+# @happyvertical/smrt-inference
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Fallback for s-m-r-t inference tiers. Bind a tier to an ordered chain of models
+(an alias) and a call that hits a quota, a bad key or a renamed model moves to
+the next model instead of failing. Every move is recorded on the call's usage
+row, and `smrt doctor --ai` checks the bindings before you need them.
+
+## Installation
+
+```bash
+pnpm add @happyvertical/smrt-inference
+```
+
+Import it once at startup. Importing registers it with `@happyvertical/smrt-core`.
+
+```ts
+import '@happyvertical/smrt-inference';
+```
+
+## Usage
+
+```ts
+import { config } from '@happyvertical/smrt-core';
+
+config({
+  inference: {
+    aliases: {
+      frontier: {
+        candidates: [
+          { provider: 'anthropic', model: 'claude-sonnet-4-5' },
+          { provider: 'openai', model: 'gpt-5' },
+          { preset: 'lemonade', model: 'qwen3-8b' },
+        ],
+      },
+    },
+    tiers: { great: { alias: 'frontier' } },
+  },
+});
+
+await article.do('Summarize this', { tier: 'great' });
+```
+
+- A failed candidate is classified as `quota`, `credential`, `missing-model`,
+  `transient` (retried once), `refusal` or `missing-capability` (the call
+  needs JSON or tools the model lacks), and the next one is tried.
+- When all of a tier's candidates fail, the call moves down to the next
+  cheaper tier. Set `inference.degrade: false` to stop that, or pass
+  `minimumTier` on the call (prompts: `ai.minimumTier`).
+- Text already streamed to the caller is never continued by another model.
+- The usage row of the answering call carries `tierResolved` and
+  `substitutions`. A call that nothing answered leaves a zero-token row with
+  the reason and throws `InferenceExhaustedError`.
+
+## Beyond fallback
+
+- **Fusions**: bind a tier to a fusion you define (`single`, `pair` or
+  `handoff` over smrt-prompts keys); callers keep asking for the tier.
+- **Bindings**: users and tenants bind tiers through stored configs
+  (`./resolver`) and the settings panel (`./svelte`, `./server`), tenants only
+  to targets on the app's allow-list; keys stay in smrt-secrets.
+- **Hosted inference**: per-user gateway keys (`./node`) and the spend gate
+  and kill switch (`./gate`).
+
+The app-author guide, with a worked example, is
+[Inference tiers, bindings and usage](../../docs/content/inference.md).
+
+## Checking bindings
+
+```bash
+smrt doctor --ai          # offline: providers, key variables, models, aliases, cycles
+smrt doctor --ai --probe  # also asks each endpoint for its model list
+```
+
+See [AGENTS.md](./AGENTS.md) for the full contract.

package/agents/fusions.md

@@ -0,0 +1,105 @@
+# Fusions (`src/fusion.ts`, #4753)
+
+`src/fusion.ts` runs `{ fusion: '<key>' }` tier targets. SMRT ships the
+interpreter and the vocabulary, never a fusion: modes are code, fusions are
+config. A fusion comes from, highest first (each replaces a lower one whole):
+the registered resolver's stored layer through core's
+`context.lookupFusion(key)` (#4800; the override resolver's app rows),
+`inference.fusions.<key>` (app config), or
+`defineFusion({ key, mode, seats, gate?, description? })` (process registry
+on `globalThis`). A seat is `{ name?, prompt, target, temperature?, output? }`:
+`prompt` is a smrt-prompts key whose resolved template (code default,
+config, app and tenant overrides, read through `context.db`) is the seat's
+system message, so tenant overrides of a seat need nothing new; `target` is
+a tier (resolved through the same resolver with the seat's prompt key, so
+per-user bindings apply), `{ alias }`, `{ candidates }` or an inline
+`{ provider, model }`, never a fusion. Each seat runs through
+`createChainClient()`, so chains, classified fallback and (tier seats only)
+the ladder apply per seat. `name` defaults to the prompt key and must be
+unique.
+
+Example (a consumer's code; SMRT does not ship it):
+
+```ts
+definePrompt({ key: 'contracts.skeptic', template: 'List the risks…' });
+definePrompt({ key: 'contracts.advocate', template: 'List the benefits…' });
+definePrompt({ key: 'contracts.synthesis', template: 'Weigh both reviews…' });
+config({ inference: {
+  tiers: { best: { fusion: 'contract-review' } },
+  fusions: { 'contract-review': {
+    mode: 'pair',
+    seats: [
+      { name: 'skeptic', prompt: 'contracts.skeptic', target: 'good' },
+      { name: 'advocate', prompt: 'contracts.advocate', target: { alias: 'frontier' } },
+      { name: 'synthesis', prompt: 'contracts.synthesis', target: 'great' },
+    ],
+    gate: { criteria: 'Both reviews agree on every material risk.',
+            sufficientWhen: { choice: 'agree', minConfidence: 0.85 } },
+  } },
+} });
+await contract.do('Summarize this contract', { tier: 'best' }); // unchanged caller
+```
+
+- **Modes and call counts** (model calls; a seat's fallback candidates are
+  substitutions of that seat, not extra seats): `single` 1 (one seat);
+  `pair` 3 (seats 0 and 1 in parallel, then seat 2 merges both outputs,
+  labelled by seat and model); `handoff` 2 (seat 0 drafts, seat 1 receives
+  the draft and answers). A mode that adds a call is a bug.
+- **Gate** (`pair`, `handoff`): one choice question through
+  `context.decide()` (core's typed-decision client, the `evaluate()` path)
+  over the outputs so far, labels `sufficientWhen.choice` and `continue`.
+  The chosen label at `minConfidence` or above skips the remaining stage,
+  recorded `skipped`; the answer is then seat 0's output (`pair`: the first
+  live one) or the draft. No decision client (`unavailable`) or a decision
+  error runs the stage. Its usage row has `seat: 'gate'`, a reserved seat
+  name.
+- **Failure semantics.** A dead seat never aborts a `pair`: the merge seat
+  is told which seat failed and why (class and message). A dead merge or
+  handoff answer seat falls back to the first live output, recorded
+  `degraded`; the handoff answer seat is told when the draft died. A run
+  whose seats all died throws `InferenceFusionError`
+  (`INFERENCE_FUSION_FAILED`, `result` holds every seat) with the run
+  recorded `failed`, never an empty answer. A seat whose setup fails (prompt
+  override read, its tier's resolver) dies like a model failure. A config
+  error, a caller abort, or a final seat that already streamed text
+  (`partial`) rethrows, after both `pair` seats have settled.
+- **JSON**: `output: 'json'` sends `responseFormat: json_object` and parses;
+  a seat whose text does not parse keeps it under `raw` (with `parseError`)
+  in `FusionSeatResult` and hands the raw text on, labelled. A candidate
+  without JSON mode runs the seat in text on that recovery path
+  (`capabilityFallback: { json: 'text' }`, #4758) instead of being skipped.
+- **Caller options**: generation options (`responseFormat`, `tools`,
+  `onProgress`, …) reach the final seat only; `signal` / `timeout` reach every
+  seat; `history` is the seat's instructions. `onFusionResult(result)`
+  receives the structured `FusionRunResult`. Only `message()` runs a fusion.
+- **Record**: every seat's usage row carries `runId`, `fusionKey`, `seat`
+  and the seat's prompt tags; `context.recordRun()` writes one
+  `_smrt_ai_runs` row (`ok | degraded | failed`; a seat is degraded when it
+  died or answered through a substitution) with identities and totals only
+  (`../../core/agents/ai-usage.md`). `smrt ai:usage --by fusion` totals runs.
+- **Validation**: `findInferenceConfigIssues()` names structural offenders
+  by path (`fusion-invalid`, `fusion-unknown`, `fusion-nested`); an
+  unregistered seat prompt key is a load error (`fusion-prompt-unknown`) from
+  `validateInferenceConfig()`, the resolver, the executor and
+  `defineFusion()`, but not the doctor (it runs without the app's prompts).
+- Seat prompts resolve through `@happyvertical/smrt-prompts` (a dependency;
+  never a peer, #1582), imported by a variable specifier from this package
+  (strict layouts find it; bundlers do not follow it, so the root stays browser-safe),
+  then core's `importOptionalDependency()` (a module registered with
+  `registerOptionalDependency()` in bundled builds).
+  `setFusionPromptResolver()` replaces it.
+- **Fusions are app-scope only.** A fusion multiplies hosted model spend
+  per call, so a tenant never defines or redefines one; with #4754's
+  allow-list a tenant may bind a tier to a listed fusion (`fusion:<n>`).
+- **Stored fusions (#4800).** Core hands the executor
+  `context.lookupFusion(key)`: the instance's (or registered) resolver's
+  optional `lookupFusion(key, request, context)`, then the merged
+  `inference.fusions`; `undefined` falls to the registry. Without it (older
+  cores) the executor reads config, then the registry, and validates at bind
+  time; with it the lookup starts at bind time and an unknown or invalid
+  fusion fails the first call before any seat runs (nothing recorded). Every
+  definition, whatever its source, is validated before it runs (structure
+  against the call's aliases, every seat prompt key); a resolver's answer
+  fails under path `resolver.fusions.<key>`. The #4752 validating wrapper
+  and the #4756 user resolver forward `lookupFusion` to their base /
+  `fallback` (stored user and tenant configs never define a fusion).

package/agents/hosted.md

@@ -0,0 +1,161 @@
+# Hosted inference (#4759, #4760)
+
+## Provisioning (`src/hosted.ts`, #4759)
+
+`provisionHostedInference({ db, tenantId, userId, budget })`,
+`revokeHostedInference({ db, tenantId, userId })`, or one
+`createHostedInferenceProvisioner({ db, secrets?, gateway?, admin?, actor? })`
+(`provision`, `revoke`, `syncBudget`, `status`, `listActive`, `listKnown`,
+`revokeTenant`, `syncTenantBudget`, `reconcileTenant`,
+`acknowledgeGatewayRevocation` to drop an owed deletion an operator did by
+hand).
+
+- **Gateway.** Admin calls go through `ai.admin` (`AIAdminInterface`) of the
+  `bifrost` / `litellm` provider of `@happyvertical/ai` >= 0.111.7
+  (happyvertical/sdk#1494); never reimplement gateway administration here.
+  `HostedGatewayAdmin` is `HOSTED_GATEWAY_ADMIN_METHODS` (`createProject`,
+  `createVirtualKey`, `updateVirtualKey`, `deleteVirtualKey`,
+  `listProjects`); a provider missing one is `HOSTED_GATEWAY_UNSUPPORTED`.
+  SMRT persists only `AIVirtualKey.id`, the non-secret handle (Bifrost id,
+  LiteLLM `token_id`); an id that equals or looks like a key value (`sk-…`)
+  is dropped on create and when older state is read, so that key counts as
+  having no id. Budget sync sends
+  `updateVirtualKey(id, { budget })` only (Bifrost refuses `models` /
+  `metadata`, LiteLLM `providerConfigs`: `ADMIN_UNSUPPORTED_FIELD`);
+  `unsupported` now means the grant has no key id. Revoke ends local use
+  before touching the gateway (config restored, state retired with the key
+  owed, then `deleteVirtualKey`; `false` is `missing`, an already revoked
+  key, nothing owed). A deletion the gateway did not perform (`failed`, a
+  key a failed provision left, `unsupported`: no admin configured or no key
+  id; entries from before 0.111.7 also read `unsupported`) goes to the
+  state's `pendingGatewayRevocations` (id + alias, never the key), is
+  audited with `manualGatewayRevocation`, and is retried by `revoke()` /
+  `reconcileTenant()`; each retry that settles one writes
+  `inference.hosted.revoke-retry` with `settled: [{ keyId, keyAlias,
+  gateway }]`, closing those records. Entries without an id stay for an
+  operator (`acknowledgeGatewayRevocation`).
+- **Config, read once from the environment** (`getHostedGatewayConfig()`;
+  unset provider = hosted off): `SMRT_INFERENCE_HOSTED_PROVIDER`
+  (`bifrost|litellm`), `_BASE_URL` (the OpenAI-compatible endpoint keys are
+  used against), `_ADMIN_URL`, `_ADMIN_KEY` (LiteLLM master key, required for
+  litellm; or a Bifrost admin key), `_ADMIN_USER` / `_ADMIN_PASSWORD`
+  (Bifrost Basic auth), `_MODEL`, `_<TIER>_MODEL`, `_MODELS` (comma list the
+  key may use; default the binding's models), `_KEY_DURATION`,
+  `_ADMIN_TIMEOUT_MS` (deadline per admin call, default 30 s, also passed as
+  the SDK `timeout`; a call past it is a failure, so a stalled gateway never
+  holds the lifecycle lock). A create that fails ambiguously (timeout,
+  transport reset, 5xx, unclassified; anything but a credential, quota,
+  missing-model or refusal answer per `classifyInferenceFailure`) records
+  its alias as owed, since the gateway may have minted the key.
+  Errors name missing fields, never values.
+- **Storage** (smrt-secrets, `context = tenantId`): the key lives only in
+  the user's `inference.provider-config:user:<id>` (`source: 'hosted'`,
+  `mode: 'hosted'`, `gatewayKeyAlias`, a binding per configured tier);
+  `inference.hosted:user:<id>` (category `inference.hosted`) holds status,
+  generation, key id and alias, budget and the user's prior config;
+  `inference.hosted:project` the tenant's gateway project (created once
+  through `runOnce()`, name `smrt tenant <tenantId>`, LiteLLM id
+  `smrt-<tenantId>-<nonce>`). Without a stored record, `listProjects({
+  tenantId })` runs first and adopts the lowest-id project with that name
+  (and id prefix on LiteLLM), audited `adopted: true`, so a crash between
+  create and persist never makes a second project; a failed lookup fails
+  the provision rather than create blind. Every write passes
+  `auditDetails.operation` (`inference.hosted.provision|revoke|budget|project`)
+  to `SecretAuditLog`.
+- **Idempotency and ordering.** An active grant whose binding is still the
+  stored one is returned (`created: false`), its budget synced when it
+  differs. The create path runs under `runOnce()` keyed by `(tenantId,
+  userId, generation)`; revoke advances the generation. Aliases
+  (`smrt-<tenant>-<user>-g<n>-<nonce>`) and LiteLLM project ids
+  (`smrt-<tenant>-<nonce>`) are unique per attempt, so a failed attempt's
+  leftovers never block the retry. Lifecycle operations run one at a time
+  per process (a process-wide chain: rare operations, and it bounds the
+  pooled connections held) and, on PostgreSQL, under a per-tenant
+  `pg_advisory_xact_lock` held in a transaction for the whole operation
+  (its writes use other pooled connections: at most three per process), so
+  a tenant-wide revoke or reconcile never misses an in-flight provision.
+  The optional `entitled(tenantId)` check runs inside that lock before a
+  key is minted (`HOSTED_NOT_ENTITLED`) and before an entitled reconcile
+  acts (a stale snapshot then revokes), so a stale gain or late retry after
+  a downgrade does not activate. Hosts running the listener or reconciler
+  across processes should set it. A failed binding write deletes or records the new
+  key and restores the previous state after the claim rolls back (no delete
+  inside the claim transaction: single-connection adapters would wait on
+  it). A user who replaced the hosted binding keeps their config: an
+  explicit `provision()` retires the stale key and mints anew; a reconcile
+  only retires it and marks the state `optedOut`, which later reconciles
+  respect until an explicit `provision()`. Writes through the #4756
+  resolver (the settings panel) do not take this lock.
+- **Revoke** restores the prior config when it holds a binding or a mode,
+  else writes `{ mode: 'off' }`; a config the user replaced is kept.
+- **Budget** is integer micro-dollars (`{ limitMicros, window? }`, `day |
+  week | month | year`, default month), converted to the gateway's USD
+  `maxLimit` and `1d|7d|30d|365d` only at the admin call. #4760's spend gate
+  is the authoritative pre-call check; the gateway budget is defense in depth.
+  An unchanged budget is never re-sent (a gateway update may restart the
+  spend window); after a failed or timed-out update (`budgetSyncUncertainAt`,
+  it may still land) syncs re-apply the stored budget until a confirmed
+  update 15 minutes later.
+- **Entitlement hook.** `createHostedInferenceEntitlementListener({
+  provisioner, users, budgetFor?, defaultBudget? })` structurally matches
+  smrt-subscriptions' `EntitlementChangeListener` (register it with
+  `registerEntitlementChangeListener()`; the host wraps its webhook write in
+  `trackEntitlementChange()`). A change touching `inference.hosted`
+  reconciles the tenant to the change's `after` snapshot through
+  `createHostedInferenceReconciler()`: entitled provisions `users(tenantId)`
+  (membership is the host's; exclude users an operator revoked) unless
+  they opted out, and syncs budgets; not entitled revokes every grant; both
+  retry owed deletions (also for active grants: they are old keys). The
+  plan's budget is the grant's `metadata.budget` (`{ limitMicros, window }`).
+  A replayed webhook diffs to "no change", so hosts also call the reconciler
+  with `snapshotEntitlements()` on redelivery and on a schedule; that is
+  what repairs a reaction lost to a crash. Per-user failures are thrown
+  together after all users were tried.
+- **Keys.** Never in results, errors (gateway failures carry only the
+  operation and error code), audit details, `runOnce()` results or logs.
+  Usage rows carry `source: hosted` and `gatewayKeyAlias` (from
+  `InferenceTarget.usageTags`). Other processes see a new binding after the
+  resolver cache expires; this process is invalidated through
+  `invalidateUserInferenceResolvers()`.
+
+## Spend gate and kill switch (`src/gate.ts`, #4760)
+
+`createHostedInferenceGate({ killSwitch, spend, principal?,
+requireSubscription?, onWarning? })` is core's `InferenceGate` (seam:
+[core inference.md](../../core/agents/inference.md), "Hosted gate");
+`registerHostedInferenceGate(gate | options)` installs it and the
+principal's user as the `userId` usage tag. Host wiring:
+
+```ts
+registerHostedInferenceGate({
+  killSwitch: new KillSwitchService({ db }),          // smrt-features
+  spend: await HostedAiSpendEvaluator.create({ db }), // smrt-subscriptions
+});
+```
+
+Both stores are structural: smrt-features and smrt-subscriptions build with
+the CLI, which depends on this package, so a dependency on either is a turbo
+cycle. Checks, in order, on every `source: hosted` call:
+
+1. **Kill switch** `inference.hosted.enabled` (`HOSTED_INFERENCE_KILL_SWITCH`):
+   global DISABLE, then the tenant's DISABLE, blocks (`policy:
+   'kill-switch'`, `disabledBy`). Uncached, no model call: **the incident
+   lever** (`smrt inference:hosted disable [--tenant <id>]`; programmatic
+   `setHostedInferenceEnabled(store, { enabled, tenantId? })`,
+   `getHostedInferenceSwitch()`). Stored as smrt-features override rows
+   with kill-switch semantics, not as a flag: a tenant ENABLE can never
+   lift a global DISABLE, and `inference.hosted` (the plan key) stays a pure
+   entitlement. Hosts with an admin UI write the same DISABLE rows through
+   `FeatureOverrideService` and their authorizer.
+2. **Tenant**: request tenant, else the principal's; none blocks (`no-tenant`).
+3. **Entitlement**: no entitled subscription blocks (`subscription`) unless
+   `requireSubscription: false`.
+4. **Spend**: the plan's tenant and per-user caps (smrt-subscriptions
+   AGENTS, "Hosted AI spend caps"): `blocked` blocks with `cap`, `spend`
+   (micro-dollars) and `resetsAt`; `warn` returns `warn` and calls
+   `onWarning` (errors swallowed). The user is the principal's, only in its
+   own tenant, and none for `userId: null` (agents).
+
+Any thrown error propagates and core blocks with `gate-error`. The plan
+check does not test the `inference.hosted` feature key: provisioning the
+hosted binding (#4759) is where entitlement to hosted is granted.

package/agents/resolver.md

@@ -0,0 +1,65 @@
+# Per-user and per-tenant bindings (`src/resolver.ts`, #4756)
+
+`createUserInferenceResolver({ db | secrets, fallback?, ttlMs?, principal? })`
+is core's `InferenceResolver`. Per tier, highest first: the user's stored
+config (its `tiers[tier]`, then its default binding), the tenant's (same
+order), `fallback`, the app `inference` block, then the configured client
+(`SMRT_AI_*` / process `ai`). Every resolution carries `target.source`
+(`byo | local | hosted | app`); core records it as the `source` usage tag.
+
+- **Registration order.** `registerUserInferenceResolver(r)` installs
+  `createInferenceResolver(r)`: the #4752 validating resolver stays outermost
+  (chain config is validated first), then the user resolver, then its
+  `fallback`, then core's config resolution. A later `registerInference()`
+  keeps it (the wrapper is marked as this package's own); a bare
+  `setInferenceResolver(r)` would drop the validation.
+- **Storage.** One smrt-secrets type, `inference.provider-config`
+  (`category`), via `SecretService.store()` / `retrieve()` only, with
+  `context = tenantId`. Names: `inference.provider-config:user:<userId>` and
+  `inference.provider-config:tenant`. The value is
+  `{ provider, baseUrl, model, apiKey, preset, params, source?, mode?,
+  gatewayKeyAlias?, tiers? }`; hosted (#4759) writes the same type with
+  `source: 'hosted'` and `gatewayKeyAlias` (refused on any other source;
+  every target of the config carries it as `usageTags`).
+  `parseInferenceProviderConfig()` requires every binding to be
+  self-contained (`baseUrl` and `apiKey`, `'smrt-local'` for keyless local
+  servers, or a preset that supplies the endpoint), so a stored binding never
+  borrows the server's environment credentials; it also refuses a `baseUrl`
+  with userinfo, query or fragment, `params` beyond the scalar generation
+  options in `INFERENCE_PROVIDER_PARAM_KEYS`, a preset without a model,
+  unknown fields, and `source: 'app'`. Redaction copies known non-secret
+  fields only. A corrupt stored value fails the call with
+  `InferenceProviderConfigError` (paths only, never values); a missing,
+  disabled or expired secret is simply no layer.
+- **Identity.** The ambient principal (tenancy context, then the users
+  session context: smrt-preferences' order). With a principal: its tenant
+  and user. A request for another tenant (an object of tenant B used by a
+  principal of A) reads nothing stored and falls through to app config,
+  unless `allowCrossTenant` (then B's tenant layer, no user). No principal
+  (jobs): the request's tenant, no user layer, never a guess.
+  `request.userId === null` (core `resolveInferenceTarget({ userId: null })`)
+  skips the user layer for clients shared beyond one user (agents); a string
+  `userId` is ignored.
+- **Cache.** Decrypted layers per `(tenantId, userId)` for `ttlMs` (30 s);
+  failed loads are evicted. `write(level, config)` / `remove(level)` store
+  for the current principal and invalidate; `invalidate()` takes all, one
+  tenant (every user), or one user. Who may write the tenant config is the
+  host server action's decision (#4754).
+- **Mode (#4757).** A stored config may carry `mode` (`byo | local | hosted
+  | off`) and may hold only a mode (no binding, no key). The user's mode,
+  else the tenant's, applies: `off` answers every tier `{ unavailable:
+  'off' }` and `mode()` (untiered calls) `{ mode: 'off', unavailable: 'off' }`;
+  `byo` / `local` / `hosted` bind only a stored binding of that `source`
+  (user, then tenant), else `unavailable: 'unconfigured'`, never app config.
+  No mode: the order above, reporting the binding's source as `mode`.
+  `mode()` returns an explicit mode, which core uses to route untiered calls
+  through `auto` (so they bind the same way).
+  `setMode(level, mode | null)` keeps the binding (deletes a config left
+  empty) and invalidates; `write()` replaces the whole config, mode
+  included. `resolveBindings()` adds `mode: { mode, layer }`. The #4752
+  wrapper forwards `mode()` only when the base has one.
+- **Keys.** Never logged or tagged. `resolveBindings()` returns the tier map
+  and both stored configs redacted by default (`hasApiKey`, no `apiKey`), the
+  shape the settings panel reads; `redact: false` is server-only.
+- Raw `getAI()` sites take the binding through core's
+  `resolveInferenceTarget()` (see [core inference.md](../../core/agents/inference.md)).