@octocodeai/freellama-mcp-server 0.10.0 → 0.12.0

package/docs/ARCHITECTURE.md

@@ -1,12 +1,16 @@
 # FreeLlama architecture
 
-FreeLlama is the management layer between an AI agent and local Ollama models. Agents offload
+FreeLlama is the management layer between an AI agent and local inference servers. Agents offload
 bounded tasks through MCP, the CLI, or the control API. FreeLlama selects eligible models,
 coordinates resources, and returns results with execution evidence.
 
+Ollama supports the complete managed model lifecycle. The [llama.cpp adapter](LLAMACPP.md) supports
+explicitly selected bounded text tasks on an operator-run server, sharing core admission and
+memory policy. Its unsupported capabilities are refused before generation.
+
 The calling agent owns task decomposition, prompts, and answer verification. FreeLlama owns managed
 admission, memory checks, residency coordination, and routing within operator-configured CPU/GPU
-backends. Ollama owns model storage, runner loading, inference engines, and token generation;
+backends. The configured inference server owns model storage, runner loading, and token generation;
 the operating system and drivers schedule physical work.
 
 ## Responsibility boundary
@@ -16,15 +20,17 @@ This is the canonical ownership rule; other guides link here.
 | Responsibility | Owner |
 |---|---|
 | Goal decomposition, prompts, system messages, task dependencies, and answer verification | Calling agent |
-| Ollama processes and endpoints, exact `--cpu-model` assignments, runtime settings, and approval of pulls, stops, and deletes | Operator |
-| Prompt templates and model defaults | Ollama model metadata or a `Modelfile` |
+| Inference processes and endpoints, exact Ollama `--cpu-model` assignments, llama.cpp limits, runtime settings, and approval of pulls, stops, and deletes | Operator |
+| Prompt templates and model defaults | Inference-server configuration; Ollama model metadata or a `Modelfile` on the Ollama path |
 | Physical CPU/GPU scheduling and memory | OS, drivers, and accelerator runtime |
-| Model files, runners, Metal, MLX, llama.cpp, and token generation | Ollama |
-| Native `/api/*` and OpenAI-compatible `/v1/*` semantics | Ollama |
-| Installed-model discovery and capability normalization | FreeLlama |
+| Model files, runner loading, inference kernels, and token generation | Configured inference server (Ollama or explicit llama.cpp adapter) |
+| Native `/api/*` semantics | Ollama |
+| OpenAI-compatible `/v1/*` semantics | Selected inference server, within its supported protocol |
+| Ollama installed-model discovery and capability normalization | FreeLlama |
 | Task routing, advertised-context compatibility, advisory K/V estimates, evidence policy, and model rejection reasons | FreeLlama |
 | Managed admission, session affinity, bounded scope history, and model-transition coordination | FreeLlama |
-| Forecast memory reservations, host-pressure holds, and requests to unload idle unpinned models | FreeLlama managed execution |
+| Forecast memory reservations and host-pressure holds | FreeLlama managed execution and native local raw Ollama loads; external raw upstreams have narrower pressure gating |
+| Requests to unload idle unpinned Ollama models | FreeLlama Ollama lifecycle coordination |
 | Explicit per-model CPU/GPU backend assignment | FreeLlama and separate Ollama processes |
 | Grounded research adapters, citations, and verification verdicts | FreeLlama MCP server |
 | Bearer authentication of every route except the static `/_freellama/ui` shell | FreeLlama; external ingress owns TLS and tenant authorization |
@@ -46,6 +52,7 @@ flowchart LR
     F -->|"managed primary task"| G
     F -->|"managed assigned task"| U["Optional CPU Ollama"]
     F -->|"/v1 for an engine's model"| O["Optional OpenAI-compatible engines<br/>LM Studio, llama.cpp, vLLM, cloud"]
+    F -->|"explicit managed text task"| LC["Operator-run local llama.cpp"]
     G --> E["Metal, MLX, CUDA, ROCm, Vulkan, or CPU runner"]
     U --> E2["CPU runner"]
 ```
@@ -72,6 +79,7 @@ flowchart TB
     subgraph clients["Clients"]
         CLI["freellama CLI"]
         MCP["MCP client"]
+        FLL["freellama CLI / gateway"]
         RUST["Embedded Rust application"]
         RAW["Ollama-compatible client"]
     end
@@ -89,21 +97,29 @@ flowchart TB
 
     CLI --> CORE
     MCP --> CORE
+    FLL -->|"in-process MCP handlers"| API
+    FLL -->|"streamed rewrites"| PX
     RUST --> CORE
     CORE --> API
     RAW --> PX
     API --> GPU
     API --> CPU
+    API -->|"explicit bounded text"| LC["Local llama.cpp"]
+    PX -->|"configured /v1 model"| EXT["OpenAI-compatible engines"]
     PX --> GPU
     MCP --> ADAPTER
     ADAPTER --> FILES
     ADAPTER --> GPU
 ```
 
-The CLI and MCP server do not reimplement routing. CLI commands other than `serve`, `doctor`,
-`init`, and benchmarking are HTTP clients of a running `freellama serve`. The MCP server reaches the
-same control API through the feature-gated NAPI boundary, and unless `FREELLAMA_MCP_AUTOSTART_SERVE=0`
+The CLI and MCP server do not reimplement routing. `freellama` commands other than `serve`, `proxy`,
+`doctor`, `init`, `catalog` (it falls back to the shipped catalog), `auth-token`, `bench-all`, and
+`policy from-eval` are HTTP clients of a running `freellama serve`. The MCP server reaches the same control API through the feature-gated NAPI boundary, and unless `FREELLAMA_MCP_AUTOSTART_SERVE=0`
 it starts the bundled `serve` in the background at launch when nothing answers on the default endpoint.
+The agent commands (`freellama run`, `chat`, `gateway`…) call the same MCP tool handlers in-process. Its gateway runs a non-streamed Ollama or OpenAI
+chat on a `freellama/<workflow|job|auto>` virtual model as managed `run_task`, rewrites streamed and
+Anthropic or OpenAI Responses requests for serve's Ollama passthrough, and proxies everything else to
+serve ([agent CLI](AGENT_CLI.md)).
 
 ## Request classification
 
@@ -113,7 +129,7 @@ The listener classifies requests by path before any model-aware work occurs:
 flowchart TD
     IN["Request to FreeLlama"] --> P{"Path starts with<br/>/_freellama/v1/?"}
     P -->|"No"| RAW["Stream through compatibility proxy"]
-    RAW --> PRIMARY["Primary Ollama"]
+    RAW --> PRIMARY["Primary Ollama or configured /v1 upstream"]
     P -->|"Yes"| C{"Control endpoint"}
     C -->|"health, machine, models"| READ["Read local and Ollama state"]
     C -->|"routes"| DECIDE["Decide without inference"]
@@ -128,6 +144,7 @@ The control API exposes these endpoints:
 | Method and path | Purpose | Inference |
 |---|---|---|
 | `GET /_freellama/v1/health` | Health, admission capacity, and configured backends | No |
+| `GET /_freellama/v1/ready` | 200 once the primary Ollama (and CPU backend, when configured) answers and its catalog reads; otherwise 503 naming what is missing | No |
 | `GET /_freellama/v1/machine` | Machine profile | No |
 | `GET /_freellama/v1/models` | Normalized installed catalog, evidence, residency, and execution backend | No |
 | `POST /_freellama/v1/routes` | Deterministic route decision | No |
@@ -139,6 +156,7 @@ The control API exposes these endpoints:
 | `POST /_freellama/v1/scopes/:id/fork` | Fork history at an expected revision | No |
 | `DELETE /_freellama/v1/scopes/:id` | Delete retained history | No |
 | `POST /_freellama/v1/warm` | Admitted load of an exact installed runner profile | Load only |
+| `POST /_freellama/v1/unload` | Unload a resident model once its running tasks finish; a model that is not loaded reports `not_resident` | No |
 | `POST /_freellama/v1/tasks` | Managed chat, vision, tools, or embedding task (body shape under [Request shapes](#request-shapes)) | Yes |
 | `POST /_freellama/v1/task-batches` | Caller-declared independent tasks with ordered per-item receipts | Yes |
 | `GET /_freellama/v1/jobs` | Deferred job metadata without prompts or results | No |
@@ -146,10 +164,16 @@ The control API exposes these endpoints:
 | `POST /_freellama/v1/jobs/:id/cancel` | Cancel one deferred task and release its local permits | No new inference |
 | `DELETE /_freellama/v1/jobs/:id` | Cancel active work, release local permits, and remove its retained record | No new inference |
 
-Status, usage, metrics, and runtime-config endpoints are listed in [monitoring](MONITORING.md).
+Status, usage, metrics, and runtime-config (`config`, `config/reload`) endpoints are listed in
+[monitoring](MONITORING.md); [API](API.md) lists every surface.
 
 ### Request shapes
 
+The fields and stream behavior below describe Ollama tasks. Explicit `name/model` requests for a
+configured llama.cpp adapter use the narrower [text contract](LLAMACPP.md#supported-contract-and-limits);
+unsupported controls are refused rather than forwarded or ignored. Its normalized receipt names
+`execution.engine:"llamacpp"` and reports physical placement as unknown.
+
 The control API uses snake_case JSON. A managed task body is the routing fields plus the payload:
 
 ```json
@@ -170,13 +194,65 @@ installed tag of that family; an unknown or ambiguous name is refused with the i
 `request_options.options`. `stop_on_repetition` (on by default when no `num_predict` caps the output,
 `false` to turn it off) streams the chat from Ollama and stops as soon
 as the output repeats its unit twice after the first copy over 96+ characters: the connection
-closes, which cancels generation upstream, and everything generated is kept with
+closes to request cancellation upstream, and everything generated is kept with
 `done_reason: "repetition"`. An Ollama "token repeat limit" abort is kept the same way
 (`done_reason: "repeat_limit"`). A stopped stream has no Ollama timing fields, so it trains no
 throughput feedback. The receipt holds `route`, `admission` (`queue_wait_ms`,
 `resource_wait_ms`, `transition_wait_ms`, `total_wait_ms`), `execution` (placement, observation,
-`resource_admission`, `context_sizing`, `memory_reservation`), `metrics`, `feedback` and
-`response`, which is Ollama's own answer (`message.content`, `eval_count`, durations).
+`resource_admission`, `context_sizing`, `memory_reservation`), `metrics`, `feedback`, `timing` and
+`response`, which is Ollama's own answer (`message.content`, `eval_count`, durations). `timing` says
+where the task's time went inside `serve`, in milliseconds: `discover_ms`, `route_ms`,
+`prepare_ms`, `admission_ms`, `dispatch_ms`, `upstream_ms` (Ollama's whole answer),
+`placement_ms`, `learning_ms`, `persist_ms`, `receipt_ms`, `total_ms`, and `overhead_ms`
+(everything except `upstream_ms`). Learned feedback is written to disk after the response
+(`feedback.persistence.persisted: "queued"`); a failed write shows in health.
+
+`stream: true` answers with server-sent events (`text/event-stream`) instead of one JSON body. The
+task uses the same routing, fallback chain, admission, memory gate and receipt as a JSON response.
+
+| Event | When | `data` |
+|---|---|---|
+| `route` | the model is chosen, before dispatch (once per fallback attempt) | `{selected_model, backend}` |
+| `admission` | queued, waiting for memory, loading, running | `{status, reason, detail?}`, the deferred-job phases |
+| `delta` | each generated piece | `{content}` or `{thinking}` |
+| `done` | finished | the receipt a non-streaming call returns |
+| `error` | refused or failed | the error body, `status` and `retry_safe` (false once a `delta` was sent) |
+
+Only generating tasks stream (`completion`, `coding`, `code_repair`, `vision`, `long_context`),
+without `defer`, `tools`, `questions`, `input` or `nearest`; anything else is refused before
+admission with 422 `stream_unsupported`. Fallback models advance only before generation starts. A
+client that disconnects aborts the task and closes the upstream request to request cancellation;
+local permit release is not proof of physical stop, and nothing is replayed. `stop_on_repetition` keeps its meaning: streaming does not turn the
+loop stop on. A deferred chat job without tools reads Ollama's output the same way and shows
+`{generated_tokens, tokens_per_second}` in its status detail (reason `generating`), which MCP
+`run_task` reports as progress.
+
+Managed Ollama responses have a byte ceiling equal to `FREELLAMA_MAX_REQUEST_BYTES` (64 MiB by
+default). It covers the complete JSON response or cumulative NDJSON bytes, including reasoning,
+tools, log probabilities, malformed frames and error bodies. Crossing it closes the response and
+returns `upstream_response_too_large` with `retry_safe:false`; any retained prefix is incomplete.
+This is a per-response wire bound, not a process-wide heap limit. The raw compatibility proxy
+continues forwarding response bytes without accumulating a complete answer.
+
+Managed SSE uses bounded queues between execution and the response body. A slow reader applies
+backpressure to generated text and route events; admission progress updates are best effort while
+the queue is full. The execution worker continues enforcing the task deadline independently of
+the reader and releases local permits on expiry. Streams without a caller deadline use the server
+task timeout. Dropping the response aborts that worker; an unread bounded result may remain until
+the client reads or closes the connection.
+
+`fallback_models` (up to 4 tags) is an explicit fallback chain run in the core
+(`platform/execution/fallback.rs`): the next model replaces `model` only when the attempt before it
+was refused with a code that proves nothing executed (`resource_admission_unavailable`,
+`context_window_exceeded`, `embedding_item_exceeds_context`, `model_exceeds_host_budget`,
+`upstream_circuit_open`, `model_not_installed`, `model_not_eligible`), and only when no earlier
+embedding slice of the same task ran. Any other error ends the chain. During each attempt the later
+chain models are registered as busy, which keeps eviction from unloading them. An attempt without a
+`model` records the model routing selected, so `attempts[].model` is never null for a routed pick and
+that model is not tried again; the usage record of a failed chain names the last model tried.
+A success adds `fallback: {used, attempts, selected}` to the receipt; a failure adds
+`fallback: {used, exhausted, attempts, note}` to the error body. A deferred job validates the chain
+at submission.
 
 A batch nests whole task bodies:
 
@@ -192,10 +268,12 @@ architecture, and disk. `memory_bytes` is portable host RAM. `memory_kind` is `u
 when host and accelerator memory are known to share a pool; host RAM must not be treated as
 discrete GPU VRAM. Ollama's `/api/ps` remains the placement authority.
 
-All other paths use the compatibility proxy. It buffers request bodies up to 64 MB so a retry can
-resend identical bytes, then streams the upstream response. It removes hop-by-hop headers, adds
-`x-freellama-proxy: 1`, always targets the primary upstream, and does not apply CPU model
-assignment. Existing Ollama clients keep predictable compatibility; clients that opt into the
+All other paths use the compatibility proxy. It buffers request bodies up to 64 MiB by default for admission
+and forwarding, then streams the upstream response. Requests are replayed only when execution
+is proven not to have started; buffering bytes does not make an inference retry safe. It removes hop-by-hop headers, adds
+`x-freellama-proxy: 1`, and does not apply CPU model assignments. Native Ollama paths target
+the primary upstream; configured OpenAI-compatible `/v1/*` requests can route by model to other
+engines, and `/v1/systemone` has asset translation. Existing Ollama clients keep predictable compatibility; clients that opt into the
 managed API receive routing and policy behavior.
 
 ## Model discovery
@@ -248,7 +326,7 @@ See [Model metadata](MODEL_SELECTION.md#model-metadata) for the public contract
 ## Deterministic routing
 
 Routing defaults to ordinary `completion`; explicit profiles refine selection. Supported task kinds are `completion`, `coding`,
-`code_repair`, `tools`, `browser`, `vision`, `embedding`, and `long_context`.
+`code_repair`, `tools`, `browser`, `vision`, `embedding`, `long_context`, and `decision`.
 
 ```mermaid
 flowchart TD
@@ -289,6 +367,13 @@ placement evidence; Ollama remains authoritative for actual runner allocation an
 
 ### Confidence evidence
 
+Ollama benchmark rankings qualify only when a schema-2 report matches the current host identity,
+observed hardware profile, exact loopback endpoint and installed model digest. Missing identity,
+legacy schema, a different machine/endpoint, or an ambiguous/mismatched model artifact supplies no
+ranking evidence. A matching throughput report still does not establish task accuracy. Quality
+policy evidence independently matches exact model digests and excludes smoke-only policies.
+See [model selection](MODEL_SELECTION.md) for calibration and quality qualification.
+
 Confidence is a grade derived from independent evidence dimensions:
 
 | Policy evidence | Benchmark evidence | Confidence | Meaning |
@@ -312,10 +397,17 @@ configuration, discovery, and shared state; extracting these owners preserves th
 Managed tasks combine routing with admission and backend-aware transition coordination. With
 `defer:true`, the server returns HTTP `202` and a process-local job handle, then runs the same task
 path. The bounded registry owns receipts and cancellation, while the existing governor owns
-admission. Batch items cannot defer. Per-task `timeout_seconds` bounds discovery, waiting, loading,
+admission. Optional durable storage retains terminal receipts without replaying unfinished work
+([deferred jobs](DEFERRED_JOBS.md)). Batch items cannot defer. Per-task `timeout_seconds` bounds discovery, waiting, loading,
 and inference together, capped by the server task timeout. See [CLI controls](CLI.md#execute-tasks)
 for retention limits and [monitoring](MONITORING.md) for the observable lifecycle.
 
+The Ollama path below includes model discovery, context sizing, transition locks and placement
+learning. The explicit [llama.cpp path](LLAMACPP.md) shares admission pools, the resource governor,
+job/deadline handling and usage accounting, while using operator-set memory and output bounds.
+It checks the ready single-model endpoint and never performs lifecycle operations, auto-selection,
+physical-placement claims or Ollama feedback learning. It sends one generation POST with no retry.
+
 ```mermaid
 flowchart TD
     T["POST /tasks or /task-batches"] --> R["Select eligible model + backend"]
@@ -326,24 +418,25 @@ flowchart TD
     B -->|"Primary"| GL{"GPU runner resident<br/>at expected identity?"}
     B -->|"CPU assignment"| CO["Add num_gpu: 0"]
     CO --> CL{"CPU runner resident<br/>at expected identity?"}
-    GL -->|"Yes"| GR["GPU shared lock"]
+    GL -->|"Yes"| GR["GPU model use lease; no load lock"]
     GL -->|"No"| GX["GPU load lock until runner observed"]
-    CL -->|"Yes"| CR["CPU shared lock"]
+    CL -->|"Yes"| CR["CPU model use lease; no load lock"]
     CL -->|"No"| CX["CPU load lock until runner observed"]
     GR --> OQ["Send to selected Ollama queue"]
     GX --> OQ
     CR --> OQ
     CX --> OQ
     OQ --> SEND["Ollama schedules bounded request"]
-    SEND --> RETRY{"500, 502,<br/>or connection-establishment failure?"}
+    SEND --> RETRY{"Connection establishment failed<br/>before execution?"}
     RETRY -->|"Retryable"| OQ
     RETRY -->|"Success"| OBS["Query selected Ollama /api/ps"]
     OBS --> OUT["Route, assignment, physical observation,<br/>admission, metrics, and response"]
-    RETRY -->|"503, timeout, or final failure"| ERR["Return upstream error"]
+    RETRY -->|"HTTP error, timeout, or uncertain failure"| ERR["Return upstream error"]
 ```
 
 Admission is independent per backend and uses weighted fair round robin: interactive gets three
-turns, normal two, and background one, preserving FIFO inside each class and preventing starvation.
+turns, normal two, and background one. Within a class, bounded model affinity can temporarily
+reorder FIFO; capacity reservation and aging bound repeated bypasses.
 The primary/GPU pool defaults to two weighted units per observed `OLLAMA_NUM_PARALLEL` slot,
 falling back to two. The optional CPU pool defaults to one. A saturated GPU burst therefore cannot
 consume the permit held for a small CPU helper. With one parallel slot, those values represent one
@@ -375,8 +468,9 @@ times, or queued for two seconds, restores FIFO order for its class. Affinity us
 own record of admitted work, never an Ollama call. Each pool's receipt counts
 `affinity_promotions`; health reports the bounds under `admission.priority_fairness.model_affinity`.
 
-A task waiting for host resources returns its execution slot after two seconds and continues
-waiting within the same deadline. It remains counted against the backend's bounded queue, so
+An Ollama task waiting for host resources returns its execution slot after at most two seconds
+of slot-held resource waiting. A llama.cpp task yields after its initial bounded telemetry probe.
+Both continue waiting within the same queue deadline and remain counted against the pool's bounded queue, so
 memory pressure cannot create an invisible backlog. Health reports slot and resource waiters
 separately. Returning to slot admission transfers the existing queue registration atomically;
 new arrivals and a smaller reloaded queue limit cannot displace accepted work. Expired deadlines
@@ -389,7 +483,8 @@ load, which takes the load lock and observes once more after winning it (a same-
 finished meanwhile is then shared).
 See [Load lock and use lease](CPU_GPU_ROUTING.md#load-lock-and-use-lease). Session affinity is bound only
 after successful upstream execution, so refused and failed tasks cannot change later routing.
-An explicit `done:false` chat response or a reported upstream error is a failure, even with HTTP 200.
+An Ollama chat response requires boolean `done:true`; missing, non-boolean or false completion
+markers and reported upstream errors are failures, even with HTTP 200.
 It cannot train runtime feedback or bind affinity. Completed failed exchanges retain the raw response
 and honor a requested immediate unload. Transport failures also attempt that unload and return its
 lifecycle receipt, but a disconnected inference response alone does not prove upstream cancellation.
@@ -399,7 +494,8 @@ data, including per-item batch failures.
 `POST /_freellama/v1/task-batches` is for caller-declared independent work only: every item has a
 stable ID and `independent:true`; dependent workflow execution is rejected before any upstream call.
 It bounds local dispatch and returns ordered per-item success/error receipts. Every child still
-passes the same global admission and transition path above.
+passes its engine's capability checks and the shared admission/resource limits. Ollama children
+also use the transition path above.
 
 The K/V preflight derives single-sequence F16 K+V bytes-per-token for any architecture whose
 metadata declares the block count and head counts. Explicit key/value dimensions take precedence
@@ -444,10 +540,11 @@ admits such a load with a `memory_advisory` instead, after unloading idle runner
 Each pressure signal recovers independently after two fresh samples that meet its recovery
 threshold. A recovered CPU-load signal therefore clears even while RAM remains low.
 
-On Linux and macOS the default `require_memory` telemetry policy holds local inference when
-available RAM is missing, even when a raw request has no model-footprint estimate. Windows and other
-platforms have no host collector, so they default to `best_effort` and admit with
-`admitted_telemetry_unknown`; elsewhere `best_effort` is an explicit opt-in;
+On Linux, macOS and Windows the default `require_memory` telemetry policy holds local inference when
+available RAM is missing, even when a raw request has no model-footprint estimate. Windows uses a
+bounded PowerShell CIM query for total and available physical RAM; its CPU load, OS-pressure,
+thermal, swap and GPU signals remain unavailable. Other platforms have no host collector, so they
+default to `best_effort` and admit with `admitted_telemetry_unknown`; elsewhere `best_effort` is an explicit opt-in;
 `require_all` additionally requires the collector's applicable load, OS-pressure, and thermal signals.
 Unsupported Linux collector signals remain explicitly unavailable rather than being fabricated.
 The pure `ResourceSnapshot::assess_capacity` calculation is shared by admission and previews.
@@ -486,7 +583,7 @@ transition path rather than claiming verified runner compatibility.
 A compatible resident runner needs no backend lock: nonempty model digest and effective context must
 match the observed runner (a request that sends no `num_ctx` also accepts a runner whose context
 `/api/ps` omits). Name-only residency is insufficient, and `admission.reason` names the degraded
-identity. Every task holds a per-model use lease for its whole request, which keeps its runner from
+identity. Every managed Ollama task holds a per-model use lease for its whole request, which keeps its runner from
 being evicted or unloaded. Forecast release invalidates the
 host sample generation so an older available-memory observation cannot spend released capacity.
 A task that must load or reload a runner takes that backend's load lock until the runner is
@@ -509,8 +606,10 @@ waits up to the raw queue wait (control API: the managed queue wait), then gets
 `queue_wait_ms`, `resource_wait_ms`, `transition_wait_ms`, and
 their sum as `total_wait_ms`. Health and route receipts expose sampled pressure and reservations.
 
-HTTP `500` and `502` status codes and connection-establishment failures use bounded retry with backoff.
-HTTP `504`, transport timeouts and post-send disconnects are not replayed.
+Connection-establishment failures use bounded retry with backoff because they precede execution.
+Managed inference and raw mutation/inference requests are not replayed after HTTP `500` or `502`: an
+Ollama error may arrive after the runner has already performed work. Raw read-only GET/HEAD status
+failures may still be retried. HTTP `504`, transport timeouts and post-send disconnects are not replayed.
 FreeLlama does not retry `503 Service Unavailable` because retrying while holding admission can
 amplify overload. It does not retry a generation timeout because the first request might still run.
 
@@ -561,16 +660,18 @@ Execution receipts expose `completion_planning`, including estimated costs and w
 influenced selection. Ollama's cached-token counts inform prefill-rate learning; FreeLlama does not
 export or manage KV tensors.
 
-Raw proxy traffic always uses the primary upstream. For setup and runtime verification, see
+Raw Ollama traffic uses the primary upstream; configured OpenAI-compatible upstreams are the
+separate `/v1/*` model-routing exception. For setup and runtime verification, see
 [Run models on CPU and GPU](CPU_GPU_ROUTING.md).
 
 ## MCP tool flow
 
 The MCP server keeps general model work (`run_task`), file-backed research (`delegate_research`, a
-confined adapter subprocess), and destructive lifecycle operations (`ollama_manage`,
-`ollama_delete`) in separate tools, so `run_task` never acquires file access and lifecycle
-operations are not hidden inside a generic action. The canonical tool table, schemas, and security
-annotations are in the [MCP server reference](https://github.com/bgauryy/FreeLlama/blob/main/packages/mcp/README.md#tools).
+confined adapter subprocess), and lifecycle operations (`ollama_manage`, `ollama_delete`) in
+separate tools. `run_task` can read caller-named `files` through its input handler; it does not gain
+the research adapter's tool loop. Lifecycle mutation is not hidden inside generic task execution. Tool schemas and security annotations
+are registered in `packages/mcp/src/tools/registrations.ts`; the tool list is in
+[API](API.md#mcp-tools). `freellama tool NAME JSON` calls the same handlers from a shell.
 
 ## Grounded research flow
 
@@ -643,7 +744,8 @@ flowchart LR
 
 Raw source, images, and embedding vectors do not need to enter the frontier model's context.
 Embedding vectors are withheld by default because they are large and not human-readable. For the
-measured accounting, see [Token economics](https://github.com/bgauryy/FreeLlama/blob/main/packages/mcp/README.md#token-economics).
+compact receipts, saved response files, and the avoided-cost estimate, see
+[Small agent context](FEATURES.md#small-agent-context).
 
 ## State and failure behavior
 
@@ -660,21 +762,27 @@ FreeLlama keeps these values in memory:
 
 Sessions are bounded (default 1024) and expire after idle time (default one hour); they store only
 model affinity, never prompts or Ollama KV. Restarting `freellama serve` clears sessions and the catalog cache. Persisted placement feedback is
-reloaded; `--ephemeral` intentionally resets it. Ollama owns model residency, so loaded
-models survive a FreeLlama restart. Scope histories and their revisions do not survive restart.
+reloaded; `--ephemeral` intentionally resets it. Inference servers own model residency, so
+restarting only FreeLlama does not itself unload their models. Scope histories and their revisions do not survive restart.
 They store messages independently from affinity and append only a successful generation at the
 expected revision. Warming uses ordinary admission and fit checks; omitted residency controls use
 a finite measured policy. See [Scope history and model warming](SCOPES_AND_WARMING.md) for the
 public contract and configured limits.
 
-In `serve`, raw generation traffic shares the primary load lock and use leases and receives
-host-pressure gating, while managed tasks also reserve forecast memory. Direct Ollama callers bypass
+In `serve`, native local raw generation traffic shares the primary load lock, use leases and governor;
+cold loads require forecast memory reservations and resident reuse receives host-pressure gating.
+External raw upstreams have a narrower pressure gate because local telemetry cannot reserve remote
+memory. Managed tasks also apply model selection and context controls. Direct Ollama callers bypass
 these controls. FreeLlama cannot preempt inference or constrain other processes. A configured but
-unreachable CPU backend makes managed catalog discovery fail closed; raw primary passthrough remains
-usable. Runtime feedback is task-specific, token-normalized, and warm-only: it can steer unpinned
+unreachable CPU backend makes managed Ollama catalog discovery fail closed; raw primary
+passthrough and explicit configured llama.cpp tasks remain usable independently of that catalog. Runtime feedback is task-specific, token-normalized, and warm-only: it can steer unpinned
 `fastest` or `balanced` work after three samples per backend and a greater-than-10% advantage, but
 cannot change CPU eligibility or steer quality routing.
 
+Admission pools, use leases and memory reservations are process-local. Sharing SQLite shares
+records and learned evidence, not a distributed execution lock or a host-wide reservation ledger.
+The governor cannot control direct engine clients or allocations made by other processes.
+
 The design fails closed when it cannot establish a required contract. Examples include an
 unreachable configured backend, an unknown confidence grade, a policy without qualified models, an
 explicit model that is not installed, or a task that exceeds the advertised context window.

package/docs/ASSESSMENT.md

@@ -0,0 +1,131 @@
+# FreeLlama assessment
+
+**Overall: 7/10 for bounded local work on a shared workstation.** FreeLlama has a coherent Rust
+management core, substantial deterministic contracts and useful local measurements. The largest
+remaining gap is evidence that local answers and tool calls are reliable across representative
+tasks and machines. The second engine works for a deliberately small text surface; release and
+hardware qualification are incomplete.
+
+These ratings were established on **2026-11-14** and reviewed again on **2026-11-15**, including
+unreleased changes. The [end-to-end Ollama review](OLLAMA_REVIEW.md) records the follow-up research,
+feature coverage, implemented repairs and current verification. Scores remain unchanged because
+regression fixes do not supply missing representative model or hardware evidence.
+Astra reviewed runtime architecture, APIs and production behavior; two Sol reviews covered
+calibration/reliability and CI/Windows/maintenance. The orchestrator reconciled their findings and
+reviewed the entry points, agent instructions and documentation. Scores are engineering judgments,
+not measured accuracy percentages, a competitive ranking, or a promise that unreleased features
+are already in npm. The overall score weighs the intended workstation use; it is not an arithmetic
+average of implemented features and optional future work.
+
+## Rating scale
+
+| Score | Meaning |
+|---|---|
+| 0 | Absent |
+| 2 | Design or scaffolding only |
+| 4 | Narrow implementation or evidence |
+| 6 | Working implementation with meaningful contracts and limited live proof |
+| 8 | Broad repeatable contracts, clear ownership and bounded live evidence; qualification gaps remain |
+| 10 | Representative repeated workloads, supported platforms and failure modes qualified in production |
+
+Half points distinguish a substantial remaining qualification gap. A high architecture score does
+not compensate for weak answer-accuracy evidence.
+
+## The five implementation priorities
+
+| Priority | Rating | Implemented and demonstrated | What keeps it below 9–10 |
+|---|---:|---|---|
+| 1. CI and test coverage | **7.5/10** | [Three-OS workflow](../.github/workflows/ci.yml), strict local Rust lint, build/type checks, 781 passing Rust tests, 1,474 passing JavaScript/evaluation checks and 51 packaging contracts | Hosted lanes have not been observed; Windows selects portable JavaScript contracts and does not qualify POSIX research/coder adapters. Full live integration, hardware and all-platform release gates remain separate. [Validation scope](TESTING.md) |
+| 2. Second engine: llama.cpp | **7/10** | Explicit loopback `name/model` text execution shares Rust admission and resource reservations; nine engine contracts and a [retained live smoke](assets/llamacpp-managed-smoke-2026-11-14.json) with Ollama unavailable | No automatic discovery/selection, streaming, tool calls, embeddings, vision, lifecycle management or observed placement for this engine. The caller supplies a bounded context/output/memory configuration. [Engine contract](LLAMACPP.md) |
+| 3. Per-machine calibration | **7/10** | Schema-2 reports bind throughput to host/hardware, local endpoint and artifact digest; job selection consumes the qualifying Rust route before catalog fallbacks | Throughput is not correctness. Capable jobs and OCR still need task-quality evidence; calibration is Ollama-specific and lacks a representative multi-machine study. Runtime/settings changes require an operator rerun. [Selection](MODEL_SELECTION.md#turn-measurements-into-routing-evidence) |
+| 4a. Accuracy of `check:` lines | **5/10** | A reproducible 30-case labeled corpus retains misses and false alarms; tool arguments receive bounded structural validation. Numeric matching and malformed tool-call handling were repaired in the follow-up | Synthetic, implementation-visible examples are regression evidence. The original source-risk recall was 25%; the repaired numeric-token checker reaches 37.5% on the same exposed cases. Injection-risk recall remains 50%, dropped-value recall 33.3%. Clean checks do not prove correctness. [Versioned corpus and limits](RELIABILITY.md#accuracy-of-result-checks) |
+| 4b. Public tool-calling evidence | **4/10** | Exact model identities, prompts, settings, raw turns and graders are retained; two models ran seven cases once, scoring 6/7 and 5/7 strictly | Ten other installed audit tags remain unmeasured. The public single-trial suite cannot rank models or estimate general reliability; the observed failures were formatting. [Table and transcripts](RELIABILITY.md#measured-on-2026-11-14) |
+| 5. Panic/lock audit and large files | **8/10** | Focused poison, sampler-panic and subprocess-cleanup coverage; production lint census found no `unwrap_used` or `await_holding_lock` hits; large production files split into helpers/tests | Thirty explicit `expect` sites were counted, not all formally proved. Lock ordering, contention and OS thread exhaustion need broader qualification. Large test modules remain. [Audit](RUST_RESOURCE_AUDIT.md) |
+
+The earlier approximate count of 426 `unwrap` calls mixed production and test code. It was a
+review starting point, not a production panic count; retaining clear test assertions is appropriate.
+Likewise, splitting files improves review boundaries without proving runtime correctness.
+
+## Other product areas
+
+| Area | Rating | Evidence and remaining constraint |
+|---|---:|---|
+| Architecture and ownership | **8/10** | Routing and resource policy stay in Rust; callers own prompts, dependencies and verification. Engine-specific operations are now explicit, but discovery, warming and feedback remain centered on Ollama. [Architecture](ARCHITECTURE.md) |
+| Resource scheduler | **8/10** | Bounded weighted queues, reservations, cancellation and resource waits have contracts; waiting tasks yield execution units while retaining bounded queue tickets. Budgets and leases are process-local, so multiple authorities do not share them. Physical GPU/host contention needs broader testing. [Monitoring](MONITORING.md), [production boundary](PRODUCTION.md) |
+| APIs and agent ergonomics | **8/10** | MCP, shell, operator CLI and HTTP reuse core contracts, receipts and explicit preview. Different engines and protocol doors support different controls; the agent CLI and global readiness still require Ollama. [API](API.md), [agent CLI](AGENT_CLI.md) |
+| Installation and packaging | **8/10** | Node 22 npm/npx installs select one of five prebuilt native packages; standalone operators can use the Rust binary without Node. Source builds need Rust. Optional dependencies must remain enabled; Intel macOS and musl Linux lack prebuilt targets, and current all-target packed-install qualification is outstanding. [Build/install](BUILD_AND_NOTES.md), [packaging checks](TESTING.md#verify-release-packages) |
+| Windows support | **6.5/10** | Bounded CIM physical-memory sampling and `require-memory` defaults have invalid/overflow/zero fixtures. Native Rust and selected portable JavaScript contracts are configured in CI but unobserved here; POSIX research/coder paths are not qualified for native Windows. CPU load, pressure, swap, thermal and GPU observations remain unknown. [Windows audit](RUST_RESOURCE_AUDIT.md#windows-memory-contract), [CI scope](TESTING.md) |
+| Research/coder boundaries | **7/10** | Read-only lookup, bounded observations and caller verification are explicit; coder is opt-in, returns a patch from an isolated copy and permits only selected commands. Real coding-outcome and sandbox-platform evidence is limited; this remains substantial scope alongside the resource manager. [Research recipe](cookbooks/research-agent.md) |
+| Performance evidence | **7/10** | Retained single-machine data covers overhead, concurrency, cold observations and selected workflows. It does not predict other hardware or broad answer quality; the historical workflow denominator is incomplete. [Benchmark](BENCHMARK.md) |
+| Deployment and recovery | **6.5/10** | Loopback/auth boundaries, bounded durable receipts, unknown-outcome recovery and refusal-before-execution fallback are documented and tested. Managed-engine changes now conflict with retained input identity without moving the lookup namespace. No complete current production gate or hardware matrix is recorded. [Production](PRODUCTION.md), [durable jobs](DEFERRED_JOBS.md) |
+| Documentation | **8/10** | Guides, recipes, agent skill, package READMEs and source examples were reconciled with code, with links and example schemas checked. Historical external claims remain dated; this was not a fresh audit of every upstream URL or a clean-machine usability study. [Index](README.md) |
+
+The research/coder boundary rating measures containment and authority, not whether those adapters
+belong in the default product. Making them a separately installed feature remains a scope and
+maintenance decision; they have not been extracted by this change.
+
+## Availability and ambitious options
+
+| Capability | Status | Consequence |
+|---|---|---|
+| Explicit fallback before execution | Implemented | Up to four caller-named candidates; attempts and refusal reasons stay in the receipt. Ambiguous inference outcomes are not replayed. [Fallback rules](FEATURES.md#fallback-chains) |
+| Engine-neutral automatic failover | **Not implemented (0/10)** | Shared admission is a foundation, but engine discovery, capabilities, health/readiness and takeover are not a complete neutral scheduler. A second text adapter alone does not provide availability failover. |
+| Local-first automatic cloud escalation after failed checks | **Not implemented (0/10)** | The caller decides whether to rerun locally or use a hosted model. Existing external-upstream transport does not supply this policy, data-transfer authorization, cost limits or correctness evaluation. |
+| Multi-machine pool | Deliberately excluded | One coordinating FreeLlama authority per inference topology remains the supported resource model. |
+
+Failover safety is stronger than failover availability: refusing to replay an uncertain POST avoids
+duplicate work, but does not make a failed request succeed. Do not describe the current fallback
+chain as automatic cross-engine recovery.
+
+For retained durable keys, changed managed-engine mappings now produce an explicit conflict.
+Original receipts remain readable; inspect them before intentionally starting work with a new key.
+The [migration and conservative invalidation rules](DEFERRED_JOBS.md#idempotency) are explicit.
+
+## Next work, in order
+
+1. **Observe CI and complete release qualification** (small to medium). Run the configured lanes,
+   packed installs on supported targets and relevant live tiers; retain their versions, results and
+   skips. Configuration and local success are not cross-platform qualification.
+2. **Broaden correctness evidence** (large). Add unseen, representative factual/extraction cases
+   with independent labels; report precision/recall and abstentions. Repeat tool cases across exact
+   model artifacts, context lengths and machines; include semantic grading alongside strict format
+   grading and preserve every attempted denominator. This is the largest route to a stronger product.
+3. **Extend the engine contract deliberately** (medium to large). Choose the next real client need
+   such as streaming or tools, add a capability matrix and failure tests, and qualify it live before
+   broadening default routing. Keep uncertain execution out of automatic replay.
+4. **Validate calibration across machines** (medium). Pair host-local performance with digest-bound
+   task-quality evaluations, document rerun triggers and measure whether picks improve actual jobs.
+5. **Finish operational qualification and scope reduction** (medium). Measure lock contention and
+   sustained load, exercise Windows CIM and supported drivers, and decide whether research/coder
+   should install separately. Prebuilt packages already remove the end-user Rust requirement.
+
+These are effort estimates. Automatic cloud escalation is optional future work after check quality
+and explicit data-transfer/cost policy are established; a multi-machine pool is outside this plan.
+
+## Historical documentation review and validation
+
+The review covered the root README/changelog/agent instructions, every operator guide, all 15
+cookbook pages, both Octocode instruction sheets, the FreeLlama skill and references, package
+guides and README generation, the benchmark harness guide, and the example policy. Five cookbook
+pages and the skill's output format required no semantic change. Raw measurement files and
+generated evidence tables were retained; their scope and denominators were corrected in the guides.
+
+Before the final 2026-11-14 documentation edits, the implementation's local macOS validation passed `yarn build`, `yarn typecheck`,
+`yarn lint:rust`, `yarn test` and `yarn test:release`. That historical snapshot included 1,292 Vitest,
+112 benchmark/grader and 10 reliability checks; one JavaScript test was skipped and four Rust
+tests ignored. A read-only macOS host-resource smoke ran separately. The retained llama.cpp and
+two-model tool-calling runs are bounded live evidence, not the full inference test suite.
+
+For this documentation pass, all 18 cookbook schema/resource integration checks and all 31
+cookbook JSON examples passed; the FreeLlama skill review reported no errors or warnings.
+The two compact Octocode sheets were checked against installed version 20.2.0: 23 query examples
+matched 12 local tool schemas, using only help/schema commands.
+The JavaScript build refreshed the packaged guides, synchronized package READMEs matched their
+source, and all 51 packaging contracts passed again. The repository-wide link pass checked 64
+Markdown files, 589 local/repository links and 178 heading anchors without errors. No new model
+inference or download was needed to update this assessment. External URLs were not rechecked.
+
+The 2026-11-15 full baseline found two failures: the Octocode documentation had changed without
+updating its embedded prompt copies. The follow-up removed that duplicate source through build-time
+Markdown imports and restored the existing equality and 30-line checks. Current validation and
+the freshly checked primary Ollama sources are recorded in the [follow-up review](OLLAMA_REVIEW.md).