@octocodeai/freellama-mcp-server 0.6.0 → 0.6.2

package/docs/ARCHITECTURE.md

@@ -27,7 +27,7 @@ This is the canonical ownership rule; other guides link here.
 | Forecast memory reservations, host-pressure holds, and requests to unload idle unpinned models | FreeLlama managed execution |
 | Explicit per-model CPU/GPU backend assignment | FreeLlama and separate Ollama processes |
 | Grounded research adapters, citations, and verification verdicts | FreeLlama MCP server |
-| Bearer authentication of every route | FreeLlama; external ingress owns TLS and tenant authorization |
+| Bearer authentication of every route except the static `/_freellama/ui` shell | FreeLlama; external ingress owns TLS and tenant authorization |
 
 FreeLlama is a governed router, not an unrestricted hardware scheduler: a CPU preference cannot move
 an arbitrary tag, and one managed call contains one task. Only the calling agent knows which tasks
@@ -92,9 +92,10 @@ flowchart TB
     ADAPTER --> GPU
 ```
 
-The CLI and MCP server do not reimplement routing. The CLI calls the core directly. The MCP server
-uses the feature-gated NAPI boundary for native operations and the control API for serve-backed
-operations.
+The CLI and MCP server do not reimplement routing. CLI commands other than `serve`, `doctor`,
+`init`, and benchmarking are HTTP clients of a running `freellama serve`. The MCP server reaches the
+same control API through the feature-gated NAPI boundary, and unless `FREELLAMA_MCP_AUTOSTART_SERVE=0`
+it starts the bundled `serve` in the background at launch when nothing answers on the default endpoint.
 
 ## Request classification
 
@@ -124,17 +125,21 @@ The control API exposes these endpoints:
 | `POST /_freellama/v1/routes` | Deterministic route decision | No |
 | `POST /_freellama/v1/sessions` | New model-affinity session | No |
 | `DELETE /_freellama/v1/sessions/:session_id` | Release model-affinity session | No |
+| `POST /_freellama/v1/sessions/:session_id/kill` | Release a session and cancel its in-flight tasks; the runner stays loaded | No new inference |
 | `POST /_freellama/v1/scopes` | Create bounded message history | No |
 | `GET /_freellama/v1/scopes/:id` | Scope metadata; optional explicit history inclusion | No |
 | `POST /_freellama/v1/scopes/:id/fork` | Fork history at an expected revision | No |
 | `DELETE /_freellama/v1/scopes/:id` | Delete retained history | No |
 | `POST /_freellama/v1/warm` | Admitted load of an exact installed runner profile | Load only |
 | `POST /_freellama/v1/tasks` | Managed chat, vision, tools, or embedding task (body shape under [Request shapes](#request-shapes)) | Yes |
+| `POST /_freellama/v1/task-batches` | Caller-declared independent tasks with ordered per-item receipts | Yes |
 | `GET /_freellama/v1/jobs` | Deferred job metadata without prompts or results | No |
 | `GET /_freellama/v1/jobs/:id` | One deferred receipt and retained result | No |
 | `POST /_freellama/v1/jobs/:id/cancel` | Cancel one deferred task and release its local permits | No new inference |
 | `DELETE /_freellama/v1/jobs/:id` | Cancel active work, release local permits, and remove its retained record | No new inference |
 
+Status, usage, metrics, and runtime-config endpoints are listed in [monitoring](MONITORING.md).
+
 ### Request shapes
 
 The control API uses snake_case JSON. A managed task body is the routing fields plus the payload:
@@ -173,7 +178,7 @@ A batch nests whole task bodies:
 `POST /sessions` with an empty object returns `session_id`; `POST /warm` takes `{"model": "<tag>"}`.
 
 The machine profile is host-derived rather than model-name-derived. It reports total physical
-memory on macOS (`sysctl`), Linux (`/proc`), and Windows (system APIs), along with CPU, OS,
+memory on macOS (`sysctl`), Linux (`/proc`), and Windows (CIM through PowerShell), along with CPU, OS,
 architecture, and disk. `memory_bytes` is portable host RAM. `unified_memory_bytes` is populated
 only when host and accelerator memory are known to share a pool; it must not be treated as discrete
 GPU VRAM. Ollama's `/api/ps` remains the placement authority.
@@ -208,7 +213,7 @@ Every field comes from the model's own metadata and from what FreeLlama observed
 name. `GET /_freellama/v1/models` adds, per model:
 
 - `context_default`: the context Ollama gives a request that leaves `num_ctx` unset (`source` is
-  `modelfile`, `config`, `ollama_context_length`, `vram_tier`, or `resident_runner`). A managed
+  `modelfile`, `config`, `ollama_context_length`, `vram_tier`, `resident_runner`, or `unknown`). A managed
   task that sends a different `num_ctx` reloads the runner.
 - `resident_context`: the `context_length` of the loaded runner from `/api/ps`.
 - `observed_loops`: per task kind, managed runs since start and how many ended in a repetition
@@ -243,7 +248,7 @@ flowchart TD
     F --> X{"Explicit model?"}
     X -->|"Yes"| E["Use exact model or refuse"]
     X -->|"No"| O{"Objective"}
-    O -->|"fastest"| S["Rank local performance,<br/>residency, and deterministic ties"]
+    O -->|"fastest"| S["Rank local benchmark,<br/>smaller size, and deterministic ties"]
     O -->|"balanced"| B["Prefer policy-qualified candidates;<br/>otherwise use eligible models with low confidence"]
     O -->|"quality"| Q["Require policy-qualified candidates,<br/>then prefer policy order"]
     E --> H["Grade advertised-context compatibility and evidence"]
@@ -264,8 +269,9 @@ Routing never ranks by a model's name or architecture. Automatic selection (fall
 session affinity) ranks a model last for a task kind when it has looped on that kind at least twice
 in at most twice as many managed runs; the decision then carries
 `repeats_on_this_task_last_resort`. An operator policy naming the model for the task, or an explicit
-`model`, always wins. No request option is added because of which model was chosen: stop sequences
-and decoding settings belong to the caller.
+`model`, always wins. No request option is hard-coded for a model: apart from an operator policy's
+per-model `model_options` (`think`, `num_predict`), stop sequences and decoding settings belong to
+the caller.
 
 `context_window_fit` means only that requested `num_ctx` fits the model's advertised context
 window. It is not a live-memory, free-VRAM, K/V-cache, thermal, or runner-admission verdict.
@@ -306,7 +312,7 @@ flowchart TD
     T["POST /tasks or /task-batches"] --> R["Select eligible model + backend"]
     R --> K["Estimate metadata-backed F16 K/V<br/>(or report unknown)"]
     K --> A{"Acquire selected backend's<br/>3:2:1 fair weighted budget"}
-    A -->|"No"| E503["503 server busy"]
+    A -->|"No"| E503["429 queue full or<br/>503 wait timeout"]
     A -->|"Yes"| B{"Execution backend"}
     B -->|"Primary"| GL{"GPU model resident<br/>under transition lock?"}
     B -->|"CPU assignment"| CO["Add num_gpu: 0"]
@@ -334,8 +340,8 @@ falling back to two. The optional CPU pool defaults to one. A saturated GPU burs
 consume the permit held for a small CPU helper. With one parallel slot, those values represent one
 ordinary chat and one embedding. Operators can tune both budgets from queue-wait and resident-memory evidence.
 After FreeLlama admission, each Ollama process applies its own `OLLAMA_MAX_QUEUE`, scheduler,
-`OLLAMA_NUM_PARALLEL`, and loaded-model limit. Raw proxy requests bypass weighted admission but
-still pass the raw concurrency cap, backend exclusion, and host-pressure gate.
+`OLLAMA_NUM_PARALLEL`, and loaded-model limit. Raw proxy generation requests bypass weighted
+admission but still pass the raw concurrency cap, backend exclusion, and host-pressure gate.
 
 | Task | Default cost |
 |---|---:|
@@ -375,9 +381,10 @@ stable ID and `independent:true`; dependent workflow execution is rejected befor
 It bounds local dispatch and returns ordered per-item success/error receipts. Every child still
 passes the same global admission and transition path above.
 
-The K/V preflight derives single-sequence F16 K+V bytes-per-token only for supported model
-architectures with sufficient metadata. Explicit key/value dimensions take precedence; sliding-window,
-shared-KV, recurrent, and unknown layouts report unknown. F16 is advisory because quantized KV can
+The K/V preflight derives single-sequence F16 K+V bytes-per-token for any architecture whose
+metadata declares the block count and head counts. Explicit key/value dimensions take precedence
+over embedding length divided by heads; sliding-window, shared-KV, recurrent, cross-attention, and
+unknown layouts report unknown. F16 is advisory because quantized KV can
 use less memory. The coarse model-file budget refuses files over 80% of local CPU/unified RAM only
 for loopback backends. Forecasts use each backend's independently attributed parallelism and KV
 cache settings where process inspection is available, with explicit fallback sources otherwise.
@@ -388,14 +395,17 @@ authority. Configured settings alone do not prove physical processor placement.
 The shared [resource governor](https://github.com/bgauryy/FreeLlama/blob/main/packages/rust-core/src/platform/resources/mod.rs) samples local
 available-memory estimates, load per logical CPU, OS memory pressure, active swap-out growth, and
 thermal throttling when reported. OS memory pressure holds every local demand; active swap-out
-growth holds new loads; CPU-load and thermal holds apply to CPU-assigned work and to cold loads
-(decoding weights is CPU work and a new heat source) but not to a bounded request on a runner
-already resident on the GPU, which neither waits for the cores nor cools the package by waiting (`ResourceDemand::is_held_by`; the assessment's
+growth holds new loads; the CPU-load hold applies only to CPU-assigned work (saturated cores slow
+nothing on the GPU, so raw passthrough, which cannot know residency, is not refused for it), and to
+all work on a host whose sampler sees no accelerator; the
+thermal hold applies to CPU-assigned work and to cold loads (a new heat source) but not to a bounded
+request on a runner already resident on the GPU (`ResourceDemand::is_held_by`; the assessment's
 `held_by` lists the holds that applied). Low available RAM (below the larger of 15% or 1 GiB, clearing after two
 healthy samples above the larger of 20% or 2 GiB) depends on the platform (`ResourcePolicy::low_memory`).
+A new load must leave that reserve free, but never more than its own size above the 1 GiB floor.
 On macOS (`advisory`) the estimate undercounts what the OS can reclaim, so a load that does not cover
-it first unloads idle runners and then dispatches with a `memory_advisory`. On Linux, Windows, and
-other platforms (`refuse`) a new load that does not fit first unloads idle runners and then waits
+it first unloads idle runners and then dispatches with a `memory_advisory`. On Linux and other
+platforms (`refuse`) a new load that does not fit first unloads idle runners and then waits
 with `held_insufficient_capacity` until a permit release or a fresh sample frees capacity, or refuses
 with 503 at its deadline: Linux `MemAvailable` already includes reclaimable cache and is the hard cap
 in a container, so over-admitting invites the OOM killer. Already-resident runners add no memory and
@@ -406,8 +416,10 @@ seconds; unavailable signals stay unknown. Remote backends bypass this host gate
 Each pressure signal recovers independently after two fresh samples that meet its recovery
 threshold. A recovered CPU-load signal therefore clears even while RAM remains low.
 
-The default `require_memory` telemetry policy holds local inference when available RAM is missing,
-even when a raw request has no model-footprint estimate. `best_effort` is an explicit opt-in;
+On Linux and macOS the default `require_memory` telemetry policy holds local inference when
+available RAM is missing, even when a raw request has no model-footprint estimate. Windows and other
+platforms have no host collector, so they default to `best_effort` and admit with
+`admitted_telemetry_unknown`; elsewhere `best_effort` is an explicit opt-in;
 `require_all` additionally requires the collector's applicable load, OS-pressure, and thermal signals.
 Unsupported Linux collector signals remain explicitly unavailable rather than being fabricated.
 The pure `ResourceSnapshot::assess_capacity` calculation is shared by admission and previews.
@@ -442,11 +454,11 @@ Resident tasks share a backend lock. Nonresident tasks take that backend's exclu
 load cannot race another managed task on the same server. CPU and GPU backends have separate locks
 and admission pools, so they can progress independently.
 
-In `serve`, mutating raw proxy requests also take the primary backend's exclusive transition lock,
-returning 503 when busy. The response body holds this guard through EOF or disconnect. Metadata
-GET/HEAD and `/api/show` requests remain available. Raw inference waits at most 500 ms for host
-pressure to clear, then returns 503 with `Retry-After`; empty `keep_alive:0` unload requests bypass
-pressure. Raw request-body collection has the configured request deadline too; an incomplete upload
+In `serve`, raw generation requests also take the primary backend's exclusive transition lock,
+returning 503 when it stays busy past the raw queue wait. The response body holds this guard through
+EOF or disconnect. Metadata GET/HEAD, `/api/show`, model-store calls (pull, push, copy, delete, and
+blob uploads), and empty `keep_alive:0` unload requests take no lock and bypass pressure. Raw
+inference waits at most two seconds for host pressure to clear, then returns 503 with `Retry-After`. Raw request-body collection has the configured request deadline too; an incomplete upload
 returns HTTP 408 and releases its guards without contacting Ollama. Admission receipts report
 `queue_wait_ms`, `resource_wait_ms`, `transition_wait_ms`, and
 their sum as `total_wait_ms`. Health and route receipts expose sampled pressure and reservations.
@@ -599,7 +611,7 @@ expected revision. Warming uses ordinary admission and fit checks; omitted resid
 a finite measured policy. See [Scope history and model warming](SCOPES_AND_WARMING.md) for the
 public contract and configured limits.
 
-In `serve`, mutating passthrough traffic shares the primary transition lock and receives
+In `serve`, raw generation traffic shares the primary transition lock and receives
 host-pressure gating, while managed tasks also reserve forecast memory. Direct Ollama callers bypass
 these controls. FreeLlama cannot preempt inference or constrain other processes. A configured but
 unreachable CPU backend makes managed catalog discovery fail closed; raw primary passthrough remains
@@ -614,8 +626,9 @@ explicit model that is not installed, or a task that exceeds the advertised cont
 ## Product boundary
 
 FreeLlama is not an inference engine, remote provider marketplace, billing layer, hosted model
-registry, general agent runtime, or A2A coordinator. It can authenticate every route with one
-operator-managed bearer token and refuses unauthenticated nonloopback listeners. An external layer
+registry, general agent runtime, or A2A coordinator. It can authenticate every route except the
+static UI shell with one operator-managed bearer token and refuses unauthenticated nonloopback
+listeners. An external layer
 must still provide TLS, tenant authorization, tenant-specific limits, and isolation.
 
 Use Ollama directly when a caller only needs raw speed from one exact model. Use FreeLlama when the

package/docs/CLI.md

@@ -2,7 +2,7 @@
 
 The `freellama` command starts and inspects the local management layer that agents use to offload
 work. It exposes diagnostics, model routing, managed execution, resource controls, benchmarking,
-and Ollama-compatible passthrough. Use the [README CLI section](https://github.com/bgauryy/FreeLlama/blob/main/README.md#or-use-the-cli) for first
+and Ollama-compatible passthrough. Use the [README CLI section](https://github.com/bgauryy/FreeLlama/blob/main/README.md#from-the-terminal) for first
 use and this reference to look up commands and controls.
 
 ## Choose a command
@@ -78,8 +78,9 @@ npx @octocodeai/freellama serve \
   --cpu-model nomic-embed-text:latest
 ```
 
-FreeLlama rejects a CPU upstream with no assignments, nonloopback upstreams, and endpoints that are
-aliases for the same socket. It also sets `options.num_gpu=0` on CPU-assigned managed requests.
+FreeLlama rejects a CPU upstream with no assignments, `--cpu-model` without `--cpu-upstream`, and a
+CPU upstream that is the same Ollama endpoint as `--upstream`, including loopback aliases on one port.
+It also sets `options.num_gpu=0` on CPU-assigned managed requests.
 Read [CPU and GPU model routing](CPU_GPU_ROUTING.md) before deploying this layout.
 
 After startup, require `contracts.placement_observation` and `placement_evidence_gate`, then inspect
@@ -145,20 +146,27 @@ FreeLlama first unloads idle unpinned runners, then dispatches anyway: the recei
 load fits. On Linux, Windows, and other platforms, where available RAM already counts reclaimable
 cache (and is the hard cap in a container), the same shortfall unloads idle runners and then holds the
 new load with `held_insufficient_capacity` until capacity returns, refusing with 503 at its deadline.
-OS memory pressure holds work on every platform. Active swapping and low RAM hold new loads only:
-an already-resident runner adds no memory and stays admitted. CPU-load (one-minute load at or above
-1.5 per logical CPU) and thermal-throttling holds apply to CPU-assigned work and to cold loads, which
-decode weights on the host cores and add a new heat source; a bounded request on a runner already
-resident on the GPU neither waits for the cores nor cools the package by waiting, so it proceeds. The receipt's `resource_admission.assessment.held_by`
+OS memory pressure holds all work where the OS reports it (macOS, and Linux when the kernel
+exposes `/proc/pressure/memory`; Windows has no host collector yet). Active swapping and low RAM hold new loads only:
+an already-resident runner adds no memory and stays admitted. The CPU-load hold (one-minute load at
+or above 1.5 per logical CPU) applies only to CPU-assigned work, which decodes on those cores; GPU
+work, loaded or not, proceeds, so another program's build never blocks a GPU model. On a host where
+the sampler sees no accelerator (no GPU device or GPU memory), every model runs on the CPU, so the
+hold applies to all work there. The
+thermal-throttling hold applies to CPU-assigned work and to cold loads, a new heat source; a bounded
+request on a runner already resident on the GPU proceeds. The receipt's `resource_admission.assessment.held_by`
 lists the holds that applied to the request.
 Both `serve` and `proxy` accept `--resource-telemetry-policy`:
 
-- `require-memory` (default): wait or refuse local inference when available RAM is unknown, including
-  raw requests with zero forecast bytes. Missing thermal data alone does not block this policy.
-- `require-all`: require all signals applicable to the collector. The built-in Linux collector does
-  not provide OS pressure or thermal signals and marks those as unsupported; custom collectors must
-  provide them. Unsupported signals are not evidence of healthy hardware.
-- `best-effort`: explicitly permit unknown telemetry unless an observed pressure hold is active.
+- `require-memory` (default on Linux and macOS): wait or refuse local inference when available RAM is
+  unknown, including raw requests with zero forecast bytes. Missing thermal data alone does not block
+  this policy.
+- `require-all`: require all signals applicable to the collector. The built-in Linux collector reads
+  OS memory pressure from `/proc/pressure/memory` when the kernel provides it, but has no thermal
+  signal and marks it as unsupported; custom collectors must provide it. Unsupported signals are not
+  evidence of healthy hardware.
+- `best-effort`: explicitly permit unknown telemetry unless an observed pressure hold is active. This
+  is the default on Windows, which has no built-in memory collector.
 
 Remote upstreams bypass local host policy. Raw metadata and empty unload requests remain available
 during a hold. Route receipts separate `queue_readiness` from `resource_readiness`, and include
@@ -320,8 +328,6 @@ The MCP server autostarts `serve` with its own environment and an arbitrary work
 put a machine-wide policy in the config directory or set the variables in the MCP client's `env`.
 `serve` prints each file it picked up.
 
-Replace `MODEL` with the directory containing your completed quality-evaluation aggregate.
-
 `policy from-eval` reads `pass_at_1`, not the throughput produced by `bench-all`. It refuses
 expired aggregates and fewer than three trials unless `--allow-smoke` is explicit, and it skips
 models that are not installed on `--ollama-endpoint` (default `FREELLAMA_OLLAMA_ENDPOINT`, then

package/docs/CPU_GPU_ROUTING.md

@@ -56,7 +56,7 @@ flowchart TB
     CP --> CPU["CPU runner"]
 ```
 
-FreeLlama does not inspect or rewrite raw passthrough request bodies. A direct Ollama client that
+FreeLlama does not rewrite raw passthrough request bodies or route them by model. A direct Ollama client that
 needs model-aware CPU placement must use the managed task endpoint instead.
 
 ## Start the backends
@@ -117,7 +117,7 @@ unsafe or ambiguous configurations:
 - A CPU model assignment without `--cpu-upstream`
 - Empty model names
 - The same socket expressed through aliases such as `localhost:11434` and `127.0.0.1:11434`
-- A nonloopback upstream
+- An upstream that points back to the FreeLlama listener, or an unspecified host such as `0.0.0.0`
 
 A nonloopback FreeLlama listener requires `--allow-remote` and bearer-token authentication.
 See the [production runbook](PRODUCTION.md) for remote ingress requirements.
@@ -216,7 +216,7 @@ curl --silent http://127.0.0.1:11436/api/ps | jq '.models[] | {name, size, size_
 A fully CPU-loaded model reports `size_vram:0`; a fully GPU-loaded model reports positive VRAM
 covering its resident `size`. Compare both values from the same `/api/ps` entry. Installed `/api/tags`
 file bytes can differ from resident memory and cannot prove a mixed placement. Positive VRAM without
-total resident size stays unknown. `observation.status` is `verified`, `mismatch`, or unavailable. Unknown, mixed, and mismatched
+total resident size stays unknown. `observation.status` is `verified`, `mismatch`, `not_resident`, or `unavailable`. Unknown, mixed, and mismatched
 samples are reported but excluded from adaptive feedback.
 
 ## Understand concurrency
@@ -252,17 +252,18 @@ FreeLlama can overlap requests across the two servers even when each Ollama proc
 `OLLAMA_NUM_PARALLEL=1`. Raising `OLLAMA_NUM_PARALLEL` affects parallel requests within one server
 and multiplies K/V-cache memory; it is a separate tuning decision. `OLLAMA_MAX_QUEUE` likewise
 bounds each Ollama process after FreeLlama admission. It does not replace either backend's weighted
-FreeLlama budget, bounded waiter count, or queue-wait deadline. Raw passthrough uses a separate
-one-stream default cap in `serve`; its permit is held until the streamed response ends or is dropped.
+FreeLlama budget, bounded waiter count, or queue-wait deadline. Raw generation passthrough uses a
+separate one-stream default cap in `serve`; its permit is held until the streamed response ends or is dropped.
 
 The runtime file's `[task_costs]` table can tune individual task-kind weights without increasing
 the entire pool's ceiling. Default vision weight four can serialize two vision requests under a
 four-unit budget even when Ollama has two slots. A lower weight requires a qualified workload;
 it does not add engine slots or relax memory checks. See
 [monitoring controls](MONITORING.md#adaptive-concurrency).
-Mutating raw requests also require exclusive access to the primary backend's managed transition
-lock and receive 503 while it is busy. Metadata GET/HEAD calls remain available. This prevents raw
-inference or model lifecycle calls from overlapping a managed generation on that backend.
+Raw generation requests also require exclusive access to the primary backend's managed transition
+lock and receive 503 if it stays busy past the raw queue wait. Metadata GET/HEAD, `/api/show`,
+model-store calls (pull, push, copy, delete), and empty `keep_alive:0` unloads remain available.
+This prevents raw inference from overlapping a managed generation on that backend.
 
 ## Understand automatic feedback
 
@@ -382,9 +383,8 @@ A configured but unreachable CPU backend causes model discovery to fail closed.
 can also be unavailable until the secondary server returns. Raw passthrough to the primary server
 continues to work because it does not depend on catalog discovery.
 
-For the surrounding runtime settings, see
-[Ollama and FreeLlama optimization](dev/OLLAMA_SYSTEM_OPTIMIZATION.md). For all `serve` flags, see the
-[CLI reference](CLI.md).
+For the surrounding Ollama settings, see [Production](PRODUCTION.md#configure-ollama-explicitly). For
+all `serve` flags, see the [CLI reference](CLI.md).
 
 ## Upstream sources
 

package/docs/MODEL_SELECTION.md

@@ -4,8 +4,9 @@ To offload a task, select an installed model with the required capabilities and
 then check its memory fit under the intended concurrency. FreeLlama applies model eligibility
 and admission; the calling agent still verifies the output.
 
-The measurements in this guide come from one Apple M4 Pro machine with 48 GB unified memory and
-are not compiled into routing. Re-run
+The measurements in this guide come from single machines and are not compiled into routing:
+[Recommended starting points](#recommended-starting-points-by-form) from an Apple M2 Pro with 32 GB,
+the sections after it from an Apple M4 Pro with 48 GB unified memory. Re-run
 `npx @octocodeai/freellama doctor`, `models`, `bench-all`, and your quality suite on the target hardware.
 `doctor.machine.memory_bytes` is host RAM; a discrete GPU's usable VRAM must be observed separately.
 
@@ -26,14 +27,55 @@ flowchart TD
     M -->|"Yes"| B["Benchmark locally and add policy evidence"]
 ```
 
+## Recommended starting points by form
+
+Measured on an Apple M2 Pro with 32 GB, Ollama 0.40.8 and FreeLlama 0.6.1 on 2026-10-30, while
+other builds kept the one-minute load near 14 on 12 cores. Every check had a known answer; the
+batteries are small, so treat these as first picks to re-check on your machine, not rankings.
+
+| Form | Model (download) | Measured | Speed (warm) | How an agent calls it |
+|---|---|---|---|---|
+| Text, tools, JSON, reasoning | `gpt-oss:20b` (13.8 GB) | 7/7: exact output, arithmetic, bat-and-ball, code reading, JSON extraction, tool call, 9/9 ticket routing | 1.4 s per routed ticket | `run_task {model:"gpt-oss:20b", prompt, think:"low"}`; add `tools` or `format:"json"` |
+| Embeddings | `nomic-embed-text:v1.5` (274 MB) | Retrieval 4/4 top-1; ticket routing 9/9 from two examples per team | 51 ms per item, batched | `run_task {task:"embedding", input:[...], returnEmbeddings:true}` |
+| Decisions (classify, route, score) | `strands-decider:2b-nvfp4` (1.3 GB) | Ticket routing 9/9, return-policy yes/no 3/3, damage rubric 2/3 | 0.17 s per question | `POST <serve>/v1/systemone {model, state, questions}`: passthrough, no MCP tool yet |
+| OCR | `glm-ocr:latest` (2.2 GB) | 4/4 phrases from a dense report strip; 0/7 on text tasks | 11 s per strip at a 400-token cap | `run_task {task:"vision", model:"glm-ocr", prompt:"Text Recognition:", images:[...], stopOnRepetition:true}` |
+
+`gpt-oss:20b` reasons before answering; the battery ran it with `think:"low"` and a 2,048-token
+output cap, so leave room for the reasoning when you set `num_predict`. `glm-ocr` reads text from images only and keeps generating
+after the visible text ends, so cap its output or set `stopOnRepetition`. The decision model needs
+Apple silicon and answers only through Ollama's decision API; reach it through `serve` so the call
+still passes FreeLlama's memory checks and stream cap (from 0.6.2 a busy CPU no longer refuses it).
+
+### Classify text three ways
+
+The same nine support tickets, routed to billing, technical or returns:
+
+| Approach | Model | Correct | Per item | Use when |
+|---|---|---:|---:|---|
+| Embeddings and per-class centroids | `nomic-embed-text:v1.5` | 9/9 | 51 ms | Many items, stable classes, a few labeled examples per class |
+| Decision model | `strands-decider:2b-nvfp4` | 9/9 | 0.5 s (first call includes the load) | You need probabilities, yes/no policy checks or rubric scores |
+| Instruction model with JSON output | `gpt-oss:20b` | 9/9 | 1.4 s | Classes change per request or need explanation |
+
+A decision API request names its questions; each answer comes back under the same name:
+
+```bash
+curl -s http://127.0.0.1:11435/v1/systemone -H 'content-type: application/json' -d '{
+  "model": "strands-decider:2b-nvfp4",
+  "state": "Why did my credit card get billed two times?",
+  "questions": {"team": {"type": "choice", "instructions": "Which team handles this ticket?",
+    "criteria": {"billing": "Charges and invoices", "technical": "App errors", "returns": "Returns and refunds"}}}
+}'
+```
+
 ## Text, code, and vision
 
 `qwen3.8:27b-mlx` was the strongest general model in the local evaluations. It handled coding,
 grounded research, image description, OCR, and summarization. Using one large multimodal model also
 avoided contention between separate heavy text and vision runners.
 
-Read the [retained model profile](https://github.com/bgauryy/FreeLlama/blob/main/skills/freellama/references/model-profile-qwen3.8-27b-mlx.md)
-and [delegation measurements](https://github.com/bgauryy/FreeLlama/blob/main/skills/freellama/references/task-delegation.md) for scope and limits.
+In those evaluations it was strong at grounded lookup, long-context retrieval and strict instruction
+following, needed `think:true` and a generous output budget for multi-step reasoning, and about a
+third of its unsupervised code-review findings were false or context-limited.
 These examples do not establish that a tag is installed or suitable on your host.
 
 Dedicated OCR models answer best to the prompts their model card documents; send them with base64

package/docs/MONITORING.md

@@ -118,8 +118,9 @@ The ledger lives next to the feedback file in the platform data directory. Choos
 `/metrics` exposes counters (`freellama_tasks_total`, `freellama_prompt_tokens_total`,
 `freellama_output_tokens_total`, `freellama_task_seconds_total`, `freellama_queue_wait_seconds_total`,
 `freellama_load_seconds_total`, `freellama_raw_requests_total`, `freellama_evictions_total`,
-`freellama_circuit_open_total`, `freellama_concurrency_limit_changes_total`) labelled by model,
-backend, task, and outcome, plus gauges for admission, queues, the raw proxy, host and GPU memory,
+`freellama_circuit_open_total`, `freellama_concurrency_limit_changes_total`); task counters are
+labelled by model, backend, task, and outcome, raw requests by kind and outcome, and the rest by
+backend. Gauges cover admission, queues, the raw proxy, host and GPU memory,
 loaded models, and `freellama_ollama_num_parallel`. Point Prometheus, Grafana Agent, or any
 OpenMetrics-compatible scraper at it.
 
@@ -132,7 +133,7 @@ FreeLlama waits for safe capacity within a deadline and bounds the retained back
 | Managed queue full | `429` | `admission_queue_full` |
 | Raw proxy over its cap after `raw_queue_wait_seconds` | `429` | |
 | Managed wait exceeded `max_queue_wait_seconds` (or the task's `max_wait_seconds`) | `503` | `admission_timeout` |
-| OS pressure, swapping, CPU, thermal, or required telemetry still holds when the waiting budget expires (low available RAM alone only warns; low RAM and swapping never hold an already-resident runner) | `503` | `resource_admission_unavailable` |
+| OS pressure, swapping, CPU, thermal, or required telemetry still holds when the waiting budget expires (low available RAM alone only warns on macOS; low RAM and swapping never hold an already-resident runner, and CPU load and thermal never hold one on the GPU) | `503` | `resource_admission_unavailable` |
 | Backend circuit open | `503` | `upstream_circuit_open` |
 
 These capacity refusals carry a `Retry-After` header and `retry_after_seconds` in the body. For a queue it
@@ -267,9 +268,10 @@ out-of-memory shrink-and-retry only applies when the context was not requested e
 
 With `context_mode = "ollama_default"` (the default), FreeLlama sizes the context as before but
 omits `num_ctx` when Ollama's own default for the model is known, covers the request, and is at most
-`auto_context_max` (32768). The default is resolved in Ollama's order: the Modelfile's `num_ctx`,
-then `OLLAMA_CONTEXT_LENGTH`, then (discrete GPU only) Ollama's VRAM tier — 4096 below 24 GiB, 32768
-below 48 GiB, 262144 above. Receipts show `context_sizing.num_ctx_sent` and the default used.
+`auto_context_max` (32768). The default is the Modelfile's `num_ctx`, then the
+runtime file's `ollama_default_context`, then `OLLAMA_CONTEXT_LENGTH`, then (primary backend
+on a discrete GPU only) Ollama's VRAM tier, from total VRAM minus `OLLAMA_GPU_OVERHEAD` — 4096 below
+23 GiB, 32768 below 47 GiB, 262144 from 47 GiB. Receipts show `context_sizing.num_ctx_sent` and the default used.
 Memory estimates always use the context Ollama can allocate. Use `"explicit"` to always
 send `num_ctx`.
 

package/docs/PRODUCTION.md

@@ -34,17 +34,18 @@ published tag:
 powershell -ExecutionPolicy Bypass -File .\scripts\install.ps1 -Version VERSION
 ```
 
-The repository has no CI or release workflows; they were removed (see git history). Build, verify,
-and publish artifacts with the [release procedure](../RELEASE.md). The publish verifier checks that
-a version tag matches the workspace version.
+The repository has no CI or release workflows; they were removed (see git history). Release from a
+clean checkout: `yarn release:version <version>`, `yarn release:build` (builds each target's
+executable and addon into `release-artifacts/<target>/`), `node scripts/assemble-release.mjs
+release-artifacts release`, then `yarn release:verify:publish`. The publish verifier checks that
+`FREELLAMA_RELEASE_TAG`, when set, matches the workspace version.
 Live E2E and hardware qualification remain separate operator checks; verify those receipts before tagging.
 
-For manual assembly, build and test every claimed target in a clean
-environment, place each executable and matching addon under `release-artifacts/<target>/`, run
-`node scripts/assemble-release.mjs release-artifacts release` and `yarn release:verify:publish`, then attach
-`release/SHA256SUMS` and binaries to the release. Publish the eight `@octocodeai/freellama-native-*` packages
-first, followed by the exact-version `@octocodeai/freellama` and `@octocodeai/freellama-mcp-server` packages. Configure
-registry credentials in the release environment; never store them in the repository.
+Attach `release/SHA256SUMS` and binaries to the release. Publish the eight `@octocodeai/freellama-native-*` packages
+first, followed by the exact-version `@octocodeai/freellama` and `@octocodeai/freellama-mcp-server` packages. Publish
+the Rust crates from the same commit with `cargo publish --workspace` (it uploads `freellama-core`
+before `freellama-cli`; run it with `--dry-run` first). Configure registry credentials (`npm login`,
+`cargo login`) in the release environment; never store them in the repository.
 
 ## Configure Ollama explicitly
 
@@ -82,7 +83,7 @@ FreeLlama and Ollama have separate queues. FreeLlama caps retained managed waite
 `--max-queued-tasks`/`--cpu-max-queued-tasks`, then waits for at most
 `--max-queue-wait-seconds`. An admitted request can then enter Ollama's internal queue,
 which is bounded by `OLLAMA_MAX_QUEUE`. Raw compatibility traffic bypasses weighted managed
-admission and enters the primary Ollama queue through a separate one-stream default cap in `serve`.
+admission and, for generation, enters the primary Ollama queue through a separate one-stream default cap in `serve`.
 Set all limits from observed latency and overload behavior; changing one does not configure the
 others.
 
@@ -91,7 +92,7 @@ The following flow shows where each production setting applies:
 ```mermaid
 flowchart LR
     C["Client"] --> FQ["FreeLlama admission<br/>weighted budget + wait deadline"]
-    FQ -->|"refused"| E503["503 server busy"]
+    FQ -->|"refused"| E503["429 or 503 server busy"]
     FQ -->|"admitted"| OQ["Ollama queue<br/>OLLAMA_MAX_QUEUE"]
     RAW["Raw /api/* or /v1/*"] --> RC["Raw stream cap<br/>default 1 in serve"]
     RC -->|"refused"| E503
@@ -102,14 +103,24 @@ flowchart LR
 
 These values are conservative starting points, not universal constants. NVIDIA, AMD, Apple, and
 CPU-only hosts must pass hardware validation before promotion. See
-[Ollama and FreeLlama optimization](dev/OLLAMA_SYSTEM_OPTIMIZATION.md) for the ownership boundary and
 [Run models on CPU and GPU](CPU_GPU_ROUTING.md) for device-specific controls.
 
+Ollama already picks the accelerator, fits the model to device memory, splits layers between CPU
+and GPU, and schedules requests; FreeLlama decides which tasks reach it. These changes do not help:
+
+- Forcing one model's layers between CPU and GPU: partial CPU offload usually lowers decode speed.
+  Run a separate CPU-only Ollama process for a second model instead.
+- Running Ollama in Docker on macOS: Docker Desktop gives it no GPU acceleration there.
+- Raising context globally: K/V memory grows with context; size `num_ctx` per request.
+- Pinning several large models in one process without the memory arithmetic: weights, runtime
+  buffers and every active context must fit together.
+- `q4_0` K/V cache as a default: it trades more quality than `q8_0` for the memory it saves.
+
 Put the values in the service manager that owns each Ollama process: launchd on macOS, systemd or
 the container definition on Linux, and the Windows service wrapper. Shell exports and
 `launchctl setenv` are useful diagnostics but are login-session state, not a reboot-persistent
-production configuration. After a real service restart, rerun `doctor` and `skills/freellama/scripts/check.sh` and
-require the same visible values before admitting work.
+production configuration. After a real service restart, rerun `freellama doctor` and `freellama status`
+and require the same visible values before admitting work.
 
 ## Create the security and state files
 
@@ -170,7 +181,8 @@ only model identifiers, scale, and sample count; they contain no prompts.
 Keep adapter context settings distinct from Ollama server context. `FREELLAMA_AGENT_NUM_CTX`
 budgets the adapter conversation; older tool observations are compacted inside that budget while
 the system prompt and original question remain pinned by default. `OLLAMA_CONTEXT_LENGTH` controls
-direct Ollama requests, and managed FreeLlama tasks send their own `num_ctx`.
+direct Ollama requests; managed FreeLlama tasks size their own context and send `num_ctx` unless
+Ollama's known default covers it.
 
 ## Verify lifecycle and placement
 
@@ -187,10 +199,10 @@ Require these health contracts before admitting traffic:
 - `placement_observation: ollama_api_ps_after_execution`
 - `placement_evidence_gate: configured_or_observed`
 
-Run `skills/freellama/scripts/check.sh` with `FREELLAMA_AUTH_TOKEN_FILE` set. It verifies the
-active versions, visible Ollama settings, persisted-feedback contract, authentication contract,
-backend topology, and binary freshness. Also inspect the service definition for any setting that
-the Ollama process does not expose to FreeLlama.
+With `FREELLAMA_AUTH_TOKEN_FILE` set, run `freellama doctor` (Ollama and CLI versions, visible Ollama
+settings, machine) and read `/_freellama/v1/health` (FreeLlama version, `security`, `feedback.persistence`,
+`backends`) and `freellama status` (backend topology, loaded models, admission). Also inspect the
+service definition for any setting that the Ollama process does not expose to FreeLlama.
 
 ## Validate every hardware class
 

package/docs/README.md

@@ -2,7 +2,7 @@
 
 FreeLlama helps agents offload bounded work to local models through Ollama. The calling agent owns
 the task and verifies the answer; FreeLlama manages model selection and resources around execution.
-Start with the [repository MCP setup](https://github.com/bgauryy/FreeLlama/blob/main/README.md#connect-your-agent-mcp) to run one managed task.
+Start with the [repository MCP setup](https://github.com/bgauryy/FreeLlama/blob/main/README.md#mcp-server) to run one managed task.
 
 ## Choose a guide
 
@@ -19,7 +19,7 @@ Start with the [repository MCP setup](https://github.com/bgauryy/FreeLlama/blob/
 | Understand raw Ollama compatibility | [Ollama sidecar](ARCHITECTURE.md#ollama-sidecar) |
 | Evaluate context offload and its cost | [Token economics](https://github.com/bgauryy/FreeLlama/blob/main/packages/mcp/README.md#token-economics) |
 | Deploy the management layer | [Production runbook](PRODUCTION.md) |
-| Run development and release checks | [Testing](TESTING.md), [release procedure](https://github.com/bgauryy/FreeLlama/blob/main/RELEASE.md) |
+| Run development and release checks | [Testing](TESTING.md), [release packages](TESTING.md#verify-release-packages) |
 | Embed the routing core | [Rust core](https://github.com/bgauryy/FreeLlama/blob/main/packages/rust-core/README.md) |
 | Use the read-only status page | `/_freellama/ui` on a running `serve` (see [Monitoring](MONITORING.md)) |
 
@@ -27,10 +27,6 @@ Start with the [repository MCP setup](https://github.com/bgauryy/FreeLlama/blob/
 
 | Goal | Source |
 |---|---|
-| Review dated findings and project comparisons | [Findings report](dev/FINDINGS_AND_POSITIONING.md) |
-| Measure inference, device activity, and thermal behavior | [Local performance testing](dev/LOCAL_PERFORMANCE_TESTING.md) |
-| Separate management controls from inference tuning | [Ollama optimization boundary](dev/OLLAMA_SYSTEM_OPTIMIZATION.md) |
-| Understand backend selection decisions | [Resource-routing ADR](dev/ADR_RESOURCE_AWARE_BACKEND_ROUTING.md) |
 
 Historical measurements describe one model, machine, and workload. Inspect current state and
 repeat the relevant checks before using them to select models or set resource limits.

package/docs/TESTING.md

@@ -24,8 +24,8 @@ yarn install
 yarn build
 ```
 
-The root build compiles the host Rust release binary, NAPI addon, and single-file MCP JavaScript
-bundle. The native `.node` addon remains external to the JavaScript bundle because it is selected
+The root build compiles the host Rust release binary, NAPI addon, and MCP JavaScript bundles
+(server and research adapters). The native `.node` addon remains external to the JavaScript bundle because it is selected
 by platform triple at runtime. Published packages resolve it from an OS/CPU/libc-specific optional
 dependency rather than compiling Rust during installation.
 
@@ -40,7 +40,6 @@ Yarn. Use the `:js` or `:rust` variants to run one side only.
 | All package unit tests (JS + Rust) | `yarn test` | Rust toolchain |
 | JavaScript unit | `yarn test:js` | None |
 | Rust contracts | `yarn test:rust` | Rust toolchain |
-| Local-agent contracts | `yarn test:agents` | None after install; builds the MCP package |
 | TypeScript | `yarn typecheck` | None after install |
 | MCP integration | `yarn test:integration` | Ollama at `FREELLAMA_OLLAMA_ENDPOINT` (default `127.0.0.1:11434`) |
 | MCP end to end | `yarn test:e2e` | Ollama, `freellama serve`, and required models |
@@ -137,11 +136,11 @@ Usage-ledger unit contracts cover ordered writes, bounded queue overflow, failed
 validation, replay, and flush behavior. `monitoring_contract` also checks that a blocked ledger path
 does not hold task completion. Persistence remains best effort, not a zero-loss promise.
 
-MCP `serve.test.ts` checks startup permission failure, retry, and child-lifecycle isolation. The
+MCP `serve.test.ts` checks startup permission failure, retry, child-lifecycle isolation, and the
+background start at launch. The
 protocol integration contracts verify that spawn failure leaves the MCP connection usable and that
 fixture-backed library requests fail on unexpected product errors. External catalog availability
-is separate from these deterministic checks. Runtime-view `app-windows.test.ts` checks legitimate
-Windows static paths alongside traversal refusal.
+is separate from these deterministic checks.
 
 Launcher fixtures use `process.execPath` to avoid an extra PATH-resolved shell wrapper. After the
 MCP fixture's real executable-permission refusal, a test-only transport launches that exact fixture
@@ -211,7 +210,8 @@ yarn release:verify:publish
 Build every claimed CLI/native target in a clean environment, assemble `SHA256SUMS`, run the full
 deterministic suite, and inspect every platform package dry run before publishing explicitly.
 Publish the eight `@octocodeai/freellama-native-*` packages before `@octocodeai/freellama` and `@octocodeai/freellama-mcp-server`, all
-at the exact same version. A successful local pack dry run proves package contents; it does not
+at the exact same version, then the crates with `cargo publish --workspace` (`--dry-run` first packages
+and compiles both crates from their published sources). A successful local pack dry run proves package contents; it does not
 prove registry publication or another hardware class.
 
 ## Test one layer while iterating