@octocodeai/freellama-mcp-server 0.5.0 → 0.6.0

package/docs/ARCHITECTURE.md

@@ -11,10 +11,14 @@ the operating system and drivers schedule physical work.
 
 ## Responsibility boundary
 
-The following table defines the ownership boundary:
+This is the canonical ownership rule; other guides link here.
 
 | Responsibility | Owner |
 |---|---|
+| Goal decomposition, prompts, system messages, task dependencies, and answer verification | Calling agent |
+| Ollama processes and endpoints, exact `--cpu-model` assignments, runtime settings, and approval of pulls, stops, and deletes | Operator |
+| Prompt templates and model defaults | Ollama model metadata or a `Modelfile` |
+| Physical CPU/GPU scheduling and memory | OS, drivers, and accelerator runtime |
 | Model files, runners, Metal, MLX, llama.cpp, and token generation | Ollama |
 | Native `/api/*` and OpenAI-compatible `/v1/*` semantics | Ollama |
 | Installed-model discovery and capability normalization | FreeLlama |
@@ -23,9 +27,33 @@ The following table defines the ownership boundary:
 | Forecast memory reservations, host-pressure holds, and requests to unload idle unpinned models | FreeLlama managed execution |
 | Explicit per-model CPU/GPU backend assignment | FreeLlama and separate Ollama processes |
 | Grounded research adapters, citations, and verification verdicts | FreeLlama MCP server |
+| Bearer authentication of every route | FreeLlama; external ingress owns TLS and tenant authorization |
 
-FreeLlama remains a sidecar rather than an Ollama fork. For the compatibility rationale, see the
-[Ollama sidecar boundary](OLLAMA_SIDECAR.md).
+FreeLlama is a governed router, not an unrestricted hardware scheduler: a CPU preference cannot move
+an arbitrary tag, and one managed call contains one task. Only the calling agent knows which tasks
+are independent.
+
+### Ollama sidecar
+
+FreeLlama is a loopback sidecar, not an Ollama fork. It talks to Ollama over its documented HTTP
+contract, which also makes a second, process-isolated CPU backend possible without changing client
+payloads:
+
+```mermaid
+flowchart LR
+    C["Ollama-compatible client"] --> F["FreeLlama loopback sidecar"]
+    F -->|"raw /api and /v1"| G["Primary Ollama"]
+    F -->|"managed primary task"| G
+    F -->|"managed assigned task"| U["Optional CPU Ollama"]
+    G --> E["Metal, MLX, CUDA, ROCm, Vulkan, or CPU runner"]
+    U --> E2["CPU runner"]
+```
+
+FreeLlama does not make an individual model decode faster. It can improve workload completion by
+avoiding unsuitable routes, reducing avoidable transitions, refusing unsupported work early, and
+overlapping independent CPU and GPU model requests when the machine benefits from that layout.
+Sources: [Ollama API](https://docs.ollama.com/api/introduction),
+[streaming](https://docs.ollama.com/api/streaming), [usage](https://docs.ollama.com/api/usage).
 
 ## System topology
 
@@ -79,8 +107,7 @@ flowchart TD
     RAW --> PRIMARY["Primary Ollama"]
     P -->|"Yes"| C{"Control endpoint"}
     C -->|"health, machine, models"| READ["Read local and Ollama state"]
-    C -->|"recommendations, routes"| DECIDE["Decide without inference"]
-    C -->|"natural-routes"| INTENT["Run intent model, then route"]
+    C -->|"routes"| DECIDE["Decide without inference"]
     C -->|"tasks"| RUN["Route, admit, and execute"]
     C -->|"sessions"| SESSION["Create in-memory affinity id"]
     C -->|"scopes"| SCOPES["Retain revision-protected messages"]
@@ -94,9 +121,7 @@ The control API exposes these endpoints:
 | `GET /_freellama/v1/health` | Health, admission capacity, and configured backends | No |
 | `GET /_freellama/v1/machine` | Machine profile | No |
 | `GET /_freellama/v1/models` | Normalized installed catalog, evidence, residency, and execution backend | No |
-| `POST /_freellama/v1/recommendations` | Installed route or reviewed installation plan | No |
 | `POST /_freellama/v1/routes` | Deterministic route decision | No |
-| `POST /_freellama/v1/natural-routes` | Schema-bound intent interpretation followed by deterministic routing | Intent model only |
 | `POST /_freellama/v1/sessions` | New model-affinity session | No |
 | `DELETE /_freellama/v1/sessions/:session_id` | Release model-affinity session | No |
 | `POST /_freellama/v1/scopes` | Create bounded message history | No |
@@ -104,12 +129,49 @@ The control API exposes these endpoints:
 | `POST /_freellama/v1/scopes/:id/fork` | Fork history at an expected revision | No |
 | `DELETE /_freellama/v1/scopes/:id` | Delete retained history | No |
 | `POST /_freellama/v1/warm` | Admitted load of an exact installed runner profile | Load only |
-| `POST /_freellama/v1/tasks` | Managed chat, vision, tools, or embedding task | Yes |
+| `POST /_freellama/v1/tasks` | Managed chat, vision, tools, or embedding task (body shape under [Request shapes](#request-shapes)) | Yes |
 | `GET /_freellama/v1/jobs` | Deferred job metadata without prompts or results | No |
 | `GET /_freellama/v1/jobs/:id` | One deferred receipt and retained result | No |
 | `POST /_freellama/v1/jobs/:id/cancel` | Cancel one deferred task and release its local permits | No new inference |
 | `DELETE /_freellama/v1/jobs/:id` | Cancel active work, release local permits, and remove its retained record | No new inference |
 
+### Request shapes
+
+The control API uses snake_case JSON. A managed task body is the routing fields plus the payload:
+
+```json
+{
+  "task": "vision", "objective": "fastest", "model": "my-vision-model",
+  "session_id": "<uuid from POST /sessions>", "required_capabilities": ["vision"],
+  "prompt": "Text Recognition:", "images": ["<base64, no data-URI prefix>"],
+  "priority": "background", "keep_alive": "5m", "max_wait_seconds": 30, "timeout_seconds": 120,
+  "stop_on_repetition": true,
+  "request_options": { "options": { "temperature": 0, "num_predict": 600 }, "format": "json", "think": false }
+}
+```
+
+`model` is read the way Ollama reads it: an exact installed tag, else `name:latest`, else the one
+installed tag of that family; an unknown or ambiguous name is refused with the installed candidates.
+`messages` (Ollama message objects, images included) replaces `prompt`; `input` serves embeddings;
+`context_tokens` owns `num_ctx` and placement owns `num_gpu`, so neither is accepted inside
+`request_options.options`. `stop_on_repetition: true` streams the chat from Ollama and stops as soon
+as the output repeats its unit twice after the first copy over 96+ characters: the connection
+closes, which cancels generation upstream, and everything generated is kept with
+`done_reason: "repetition"`. An Ollama "token repeat limit" abort is kept the same way
+(`done_reason: "repeat_limit"`). A stopped stream has no Ollama timing fields, so it trains no
+throughput feedback. The receipt holds `route`, `admission` (`queue_wait_ms`,
+`resource_wait_ms`, `transition_wait_ms`, `total_wait_ms`), `execution` (placement, observation,
+`resource_admission`, `context_sizing`, `memory_reservation`), `metrics`, `feedback` and
+`response`, which is Ollama's own answer (`message.content`, `eval_count`, durations).
+
+A batch nests whole task bodies:
+
+```json
+{ "max_parallelism": 2, "tasks": [ { "id": "item-1", "independent": true, "task": { "task": "vision", "model": "my-vision-model", "prompt": "...", "images": ["..."] } } ] }
+```
+
+`POST /sessions` with an empty object returns `session_id`; `POST /warm` takes `{"model": "<tag>"}`.
+
 The machine profile is host-derived rather than model-name-derived. It reports total physical
 memory on macOS (`sysctl`), Linux (`/proc`), and Windows (system APIs), along with CPU, OS,
 architecture, and disk. `memory_bytes` is portable host RAM. `unified_memory_bytes` is populated
@@ -117,7 +179,10 @@ only when host and accelerator memory are known to share a pool; it must not be
 GPU VRAM. Ollama's `/api/ps` remains the placement authority.
 
 All other paths use the compatibility proxy. It buffers request bodies up to 64 MB so a retry can
-resend identical bytes, then streams the upstream response. It does not apply CPU model assignment.
+resend identical bytes, then streams the upstream response. It removes hop-by-hop headers, adds
+`x-freellama-proxy: 1`, always targets the primary upstream, and does not apply CPU model
+assignment. Existing Ollama clients keep predictable compatibility; clients that opt into the
+managed API receive routing and policy behavior.
 
 ## Model discovery
 
@@ -131,12 +196,26 @@ flowchart LR
     BENCH["Local benchmark report"] --> CAT
     POLICY["Task policy"] --> CAT
     ASSIGN["CPU model assignments"] --> CAT
-    CAT --> MODEL["CatalogModel<br/>capabilities · context · residency<br/>benchmark · policy rank · backend"]
+    LOOPS["Observed repetition loops"] --> CAT
+    CAT --> MODEL["CatalogModel<br/>capabilities · context · residency<br/>benchmark · policy rank · backend · loops"]
 ```
 
 FreeLlama fetches tags and residency from each configured backend. Models assigned to CPU are
 removed from the primary catalog view and included only from the secondary backend. Static catalog
-metadata is cached for 30 seconds; residency is refreshed before routing.
+metadata is cached for 30 seconds; residency and observed loops are refreshed before routing.
+
+Every field comes from the model's own metadata and from what FreeLlama observed, never from its
+name. `GET /_freellama/v1/models` adds, per model:
+
+- `context_default`: the context Ollama gives a request that leaves `num_ctx` unset (`source` is
+  `modelfile`, `config`, `ollama_context_length`, `vram_tier`, or `resident_runner`). A managed
+  task that sends a different `num_ctx` reloads the runner.
+- `resident_context`: the `context_length` of the loaded runner from `/api/ps`.
+- `observed_loops`: per task kind, managed runs since start and how many ended in a repetition
+  loop (`{runs, loops}`); process-local, not persisted.
+- `kv_cache_bytes_per_token_f16`: estimated from the declared architecture's block count, KV head
+  count and key/value lengths, for any architecture; `null` when the metadata declares recurrent,
+  state-space, sliding-window, shared or cross-attention caches, which the dense formula misstates.
 
 A failed `/api/show` response skips that model instead of failing the entire catalog. An unreachable
 configured backend fails catalog discovery closed because FreeLlama cannot prove the assigned model's
@@ -150,7 +229,7 @@ matches exact tags and digest prefixes, preserves local supported features, and
 source excerpts and usage notes. Its bounded in-memory cache expires successes after one hour and
 failures after one minute. Failed or unmatched lookups leave local inventory and evidence intact.
 Public claims do not enter the Rust catalog, task policy, quality confidence, or routing decisions.
-See [Ollama model metadata](MODEL_METADATA.md) for the public contract and request bounds.
+See [Model metadata](MODEL_SELECTION.md#model-metadata) for the public contract and request bounds.
 
 ## Deterministic routing
 
@@ -176,8 +255,17 @@ flowchart TD
     G -->|"Yes or unset"| D["RouteDecision"]
 ```
 
-Explicit model selection never substitutes another model. Every rejected candidate includes a
-reason so callers can distinguish capability, context, policy, and installation failures.
+Explicit model selection never substitutes another model. An explicit `model` is read the way
+Ollama reads it: the exact tag, else `name:latest`, else the one installed tag of that family; an
+unknown or ambiguous name is refused with the installed candidates. Every rejected candidate
+includes a reason so callers can distinguish capability, context, policy, and installation failures.
+
+Routing never ranks by a model's name or architecture. Automatic selection (fallback ranking and
+session affinity) ranks a model last for a task kind when it has looped on that kind at least twice
+in at most twice as many managed runs; the decision then carries
+`repeats_on_this_task_last_resort`. An operator policy naming the model for the task, or an explicit
+`model`, always wins. No request option is added because of which model was chosen: stop sequences
+and decoding settings belong to the caller.
 
 `context_window_fit` means only that requested `num_ctx` fits the model's advertised context
 window. It is not a live-memory, free-VRAM, K/V-cache, thermal, or runner-admission verdict.
@@ -198,37 +286,12 @@ Confidence is a grade derived from independent evidence dimensions:
 `minConfidence: "medium"` refuses before generation when either input is missing. The gate lives in
 `select_route`, so the HTTP API, CLI, MCP server, and embedded core share the same behavior.
 
-## Natural-language routing
-
-Natural-language routing cannot select a model directly:
-
-```mermaid
-sequenceDiagram
-    participant C as Client
-    participant F as FreeLlama
-    participant I as Intent model
-    participant R as Deterministic router
-
-    C->>F: Natural-language request
-    F->>I: Strict JSON schema and bounded profile
-    I-->>F: Task, objective, context, tool, and vision fields
-    F->>F: Apply deterministic guards
-    F->>R: Normalized RouteInput
-    R-->>F: RouteDecision
-    F-->>C: Intent, adjustments, and route
-```
-
-Word-boundary guards prevent incidental substrings such as `photosynthesis` from creating a vision
-requirement. Explicit requirements in the original text override weak or unsupported inferred fields.
-
-The natural route endpoint does not run the selected task. Submit a managed task to execute it.
-
 ## Managed task execution
 
-The implementation separates [admission scheduling](../packages/rust-core/src/platform/admission.rs),
-[execution coordination](../packages/rust-core/src/platform/execution.rs),
-[read-only readiness](../packages/rust-core/src/platform/readiness.rs), and
-[typed errors](../packages/rust-core/src/platform/error.rs). The platform module wires HTTP endpoints,
+The implementation separates [admission scheduling](https://github.com/bgauryy/FreeLlama/blob/main/packages/rust-core/src/platform/admission.rs),
+[execution coordination](https://github.com/bgauryy/FreeLlama/blob/main/packages/rust-core/src/platform/execution/mod.rs),
+[read-only readiness](https://github.com/bgauryy/FreeLlama/blob/main/packages/rust-core/src/platform/readiness.rs), and
+[typed errors](https://github.com/bgauryy/FreeLlama/blob/main/packages/rust-core/src/platform/error.rs). The platform module wires HTTP endpoints,
 configuration, discovery, and shared state; extracting these owners preserves the public routing API.
 
 Managed tasks combine routing with admission and backend-aware transition coordination. With
@@ -322,16 +385,26 @@ Process observations refresh on requests after five seconds; measurements made u
 settings cannot supply an exact-fit estimate. Ollama remains the live loader and final memory
 authority. Configured settings alone do not prove physical processor placement.
 
-The shared [resource governor](../packages/rust-core/src/platform/resources.rs) samples local
+The shared [resource governor](https://github.com/bgauryy/FreeLlama/blob/main/packages/rust-core/src/platform/resources/mod.rs) samples local
 available-memory estimates, load per logical CPU, OS memory pressure, active swap-out growth, and
-thermal throttling when reported. It holds new local work below the larger of 15% available RAM or
-1 GiB, and requires two healthy samples above the larger of 20% or 2 GiB to resume. Historical swap
-occupancy alone is not pressure. Samples are cached for two seconds; unavailable signals stay unknown.
-Remote backends bypass this host gate.
+thermal throttling when reported. OS memory pressure holds every local demand; active swap-out
+growth holds new loads; CPU-load and thermal holds apply to CPU-assigned work and to cold loads
+(decoding weights is CPU work and a new heat source) but not to a bounded request on a runner
+already resident on the GPU, which neither waits for the cores nor cools the package by waiting (`ResourceDemand::is_held_by`; the assessment's
+`held_by` lists the holds that applied). Low available RAM (below the larger of 15% or 1 GiB, clearing after two
+healthy samples above the larger of 20% or 2 GiB) depends on the platform (`ResourcePolicy::low_memory`).
+On macOS (`advisory`) the estimate undercounts what the OS can reclaim, so a load that does not cover
+it first unloads idle runners and then dispatches with a `memory_advisory`. On Linux, Windows, and
+other platforms (`refuse`) a new load that does not fit first unloads idle runners and then waits
+with `held_insufficient_capacity` until a permit release or a fresh sample frees capacity, or refuses
+with 503 at its deadline: Linux `MemAvailable` already includes reclaimable cache and is the hard cap
+in a container, so over-admitting invites the OOM killer. Already-resident runners add no memory and
+are admitted in both modes, also during an active-swapping hold, so a large load never locks out
+reuse of its own runner; swapping still holds new loads. Historical swap occupancy alone is not pressure. Samples are cached for two
+seconds; unavailable signals stay unknown. Remote backends bypass this host gate.
 
 Each pressure signal recovers independently after two fresh samples that meet its recovery
-threshold. A recovered CPU-load signal therefore clears even while RAM remains low. That keeps
-unchanged resident runners eligible under a memory-only hold while cold loads continue waiting.
+threshold. A recovered CPU-load signal therefore clears even while RAM remains low.
 
 The default `require_memory` telemetry policy holds local inference when available RAM is missing,
 even when a raw request has no model-footprint estimate. `best_effort` is an explicit opt-in;
@@ -339,15 +412,15 @@ even when a raw request has no model-footprint estimate. `best_effort` is an exp
 Unsupported Linux collector signals remain explicitly unavailable rather than being fabricated.
 The pure `ResourceSnapshot::assess_capacity` calculation is shared by admission and previews.
 Previews use the same footprint lookup, subtract existing reservations, and expose both queue and
-resource readiness. An available weighted slot cannot by itself produce `runnable_now` when model
-headroom is insufficient. Preview capacity is advisory and conservative, never a reservation.
+resource readiness. Insufficient model headroom reports `runnable_low_memory_estimate` and admits
+no siblings on macOS; elsewhere it reports `held_insufficient_capacity`. Preview capacity is advisory and conservative, never a reservation.
 
 Managed CPU/unified-memory tasks reserve estimated additional model-file and known F16 KV bytes
 across both backends. Verified runner footprints replace that estimate only for the same backend,
 immutable model digest, and equal-or-larger observed context. Exact resident digest/context matches
-avoid double-counting existing allocations. The reservation is rechecked after transition locking;
-an increased requirement that no longer fits refuses with 503, without waiting while holding the
-backend lock. Permits release on completion or cancellation. On discrete-GPU hosts, observed free
+avoid double-counting existing allocations. The reservation is rechecked after transition locking
+without waiting while holding the backend lock; an increased requirement that no longer fits the
+estimate dispatches with a `memory_advisory`, and a pressure hold refuses with 503. Permits release on completion or cancellation. On discrete-GPU hosts, observed free
 VRAM discounts the estimated host-memory requirement by the bytes the GPU can hold. Missing VRAM
 telemetry reserves the full estimated footprint in host RAM. The governor cannot constrain external
 allocations or preempt an already-running model.
@@ -418,35 +491,18 @@ cold-load cost do not poison the comparison. It requires more than a 10% advanta
 break a tie before the sample threshold. Quality routing, explicit model selection, and session
 affinity do not follow the latency loop.
 
-The same selection function supplies preview/recommendation receipts, intent-model execution, and
-managed task execution. This keeps advertised placement and actual request routing aligned.
+The same selection function supplies preview receipts and managed task execution. This keeps advertised placement and actual request routing aligned.
 
 Raw proxy traffic always uses the primary upstream. For setup and runtime verification, see
 [Run models on CPU and GPU](CPU_GPU_ROUTING.md).
 
 ## MCP tool flow
 
-The MCP server keeps general model work separate from file-backed research and destructive lifecycle
-operations:
-
-```mermaid
-flowchart TD
-    MC["MCP client"] --> TOOL{"Tool"}
-    TOOL -->|"doctor"| NAPI["Native core diagnostics"]
-    TOOL -->|"models"| MIX["Control API or direct Ollama/library HTTP"]
-    TOOL -->|"run_task"| TASK["Control API route or task"]
-    TOOL -->|"task_jobs"| JOB["Deferred receipts and per-task cancellation"]
-    TOOL -->|"ollama_manage"| LIFE["Direct Ollama pull or unload"]
-    TOOL -->|"ollama_delete"| DELETE["Explicit destructive Ollama delete"]
-    TOOL -->|"delegate_research"| AD["Confined adapter subprocess"]
-    AD --> FS["Allowlisted workspace"]
-    AD --> TASK
-    AD --> VER["Answer, citations, usage,<br/>and verification verdict"]
-```
-
-The tool split prevents `run_task` from acquiring file access and prevents lifecycle operations from
-being hidden inside a generic action. For schemas and security annotations, see the
-[MCP server reference](../packages/mcp/README.md).
+The MCP server keeps general model work (`run_task`), file-backed research (`delegate_research`, a
+confined adapter subprocess), and destructive lifecycle operations (`ollama_manage`,
+`ollama_delete`) in separate tools, so `run_task` never acquires file access and lifecycle
+operations are not hidden inside a generic action. The canonical tool table, schemas, and security
+annotations are in the [MCP server reference](https://github.com/bgauryy/FreeLlama/blob/main/packages/mcp/README.md#tools).
 
 ## Grounded research flow
 
@@ -499,8 +555,8 @@ Adapter cache totals remain null if any turn is unreported, with partial-report
 See [Ollama usage](https://docs.ollama.com/api/usage) and
 [the pinned prefix-cache implementation](https://github.com/ggml-org/llama.cpp/blob/74a7c897f049c17e7080423aa2111776eff6ebbf/tools/server/server-context.cpp#L3201).
 
-For adapter implementation, defaults, and measured caveats, see the
-[local adapter reference](../benchmark/local/docs/07-adapter-contracts.md).
+Adapter implementations and defaults live in
+[`packages/mcp/src/research/`](../packages/mcp/src/research/).
 
 ## Context and token boundary
 
@@ -519,7 +575,7 @@ flowchart LR
 
 Raw source, images, and embedding vectors do not need to enter the frontier model's context.
 Embedding vectors are withheld by default because they are large and not human-readable. For the
-measured accounting, see [Token economics](ECONOMICS.md).
+measured accounting, see [Token economics](https://github.com/bgauryy/FreeLlama/blob/main/packages/mcp/README.md#token-economics).
 
 ## State and failure behavior
 
@@ -543,6 +599,14 @@ expected revision. Warming uses ordinary admission and fit checks; omitted resid
 a finite measured policy. See [Scope history and model warming](SCOPES_AND_WARMING.md) for the
 public contract and configured limits.
 
+In `serve`, mutating passthrough traffic shares the primary transition lock and receives
+host-pressure gating, while managed tasks also reserve forecast memory. Direct Ollama callers bypass
+these controls. FreeLlama cannot preempt inference or constrain other processes. A configured but
+unreachable CPU backend makes managed catalog discovery fail closed; raw primary passthrough remains
+usable. Runtime feedback is task-specific, token-normalized, and warm-only: it can steer unpinned
+`fastest` or `balanced` work after three samples per backend and a greater-than-10% advantage, but
+cannot change CPU eligibility or steer quality routing.
+
 The design fails closed when it cannot establish a required contract. Examples include an
 unreachable configured backend, an unknown confidence grade, a policy without qualified models, an
 explicit model that is not installed, or a task that exceeds the advertised context window.

package/docs/CLI.md

@@ -2,7 +2,7 @@
 
 The `freellama` command starts and inspects the local management layer that agents use to offload
 work. It exposes diagnostics, model routing, managed execution, resource controls, benchmarking,
-and Ollama-compatible passthrough. Use the [README quick start](../README.md#quick-start) for first
+and Ollama-compatible passthrough. Use the [README CLI section](https://github.com/bgauryy/FreeLlama/blob/main/README.md#or-use-the-cli) for first
 use and this reference to look up commands and controls.
 
 ## Choose a command
@@ -12,13 +12,12 @@ flowchart TD
     Q{"What do you need?"}
     Q -->|"Prepare first use"| I["init"]
     Q -->|"Inspect this machine"| D["doctor or machine"]
-    Q -->|"Inspect or choose a model"| M["models, route, or recommend"]
-    Q -->|"Execute a managed request"| T["session, natural-route, or task"]
+    Q -->|"Inspect or choose a model"| M["models or route"]
+    Q -->|"Execute a managed request"| T["session or task"]
     Q -->|"Run the control plane"| S["serve"]
     Q -->|"Preserve only Ollama APIs"| P["proxy"]
-    Q -->|"Measure or compare"| B["bench-all, run, or eval"]
-    Q -->|"Create a routing policy"| E["policy-from-eval"]
-    Q -->|"Compare CLI and MCP"| L["tools"]
+    Q -->|"Measure models"| B["bench-all"]
+    Q -->|"Write a routing policy"| PE["policy from-eval"]
 ```
 
 | Command | Purpose | Needs `freellama serve` |
@@ -35,17 +34,12 @@ flowchart TD
 | `scope` | Create, inspect, fork, or delete bounded message history | Yes |
 | `warm` | Warm an installed model through managed admission | Yes |
 | `route` | Choose a model and request profile without executing it | Yes |
-| `recommend` | Return an installed route or a reviewed installation plan | Yes |
-| `natural-route` | Convert natural language to a route intent locally, then route it | Yes |
 | `task` | Route and execute one nonstreaming task | Yes |
 | `jobs` | List, inspect, cancel, or remove deferred tasks and warming operations | Yes |
 | `proxy` | Run only the Ollama-compatible retry and telemetry sidecar | No |
 | `bench-all` | Measure installed models by capability group | No |
-| `policy-from-eval` | Generate policy from quality-evaluation pass rates | No |
-| `tools` | Print MCP tools and their CLI equivalents | No |
+| `policy from-eval` | Write a task policy (`platform.toml`) from a quality-evaluation aggregate and the installed tags | No |
 | `doctor` | Inspect Ollama, hardware, versions, and effective settings | No |
-| `run` | Run a frozen suite against one Ollama build | No |
-| `eval` | Compare the same frozen suite against stock and candidate builds | No |
 
 Run `npx @octocodeai/freellama <command> --help` for every accepted flag and enum value. The executable's help
 is authoritative; this page explains how the commands fit together.
@@ -53,13 +47,16 @@ is authoritative; this page explains how the commands fit together.
 ## Start the control plane
 
 ```bash
-npx @octocodeai/freellama serve --recommendation-catalog recommendations.example.toml
+npx @octocodeai/freellama serve
 ```
 
 The default listener is `http://127.0.0.1:11435`, and the default Ollama upstream is
 `http://127.0.0.1:11434`. Use `--listen` and `--upstream` to change them. Nonloopback listeners
 require `--allow-remote` and a token from `--auth-token-file` or
-`FREELLAMA_AUTH_TOKEN_FILE`. Authentication applies to managed and passthrough routes.
+`FREELLAMA_AUTH_TOKEN_FILE`. Authentication applies to managed and passthrough routes. The token
+file must be a regular file, not a symlink, and hold one token of at least 32 bytes; surrounding
+whitespace is trimmed. On Unix it must also be owner-only (mode 0600). Windows file permissions
+(ACLs) are not checked, so protect the file with your user profile's permissions there.
 
 `serve` persists bounded adaptive feedback under the platform data directory by default. Override
 the path with `--feedback-file`; use `--ephemeral-feedback` only for disposable runs. See the
@@ -98,8 +95,8 @@ placement.
 
 ### Bound admission
 
-`--max-concurrent-tasks` is the primary/GPU cost budget, not a request count. Without a flag,
-environment variable, or runtime-file value it defaults to 2 units per `OLLAMA_NUM_PARALLEL` slot of
+`--max-concurrent-tasks` is the primary/GPU cost budget, not a request count. Without a flag or
+runtime-file value it defaults to 2 units per `OLLAMA_NUM_PARALLEL` slot of
 the Ollama server (read from its process environment where visible; Ollama's own default is 1, so 2).
 `freellama config` shows the effective value and where it came from.
 `--cpu-max-concurrent-tasks` controls the independent CPU pool and defaults to 1. Default base costs
@@ -133,19 +130,27 @@ context sizing, can also live in a runtime file (`--runtime-config`) that `serve
 changes. See [Monitoring and live tuning](MONITORING.md). The standalone
 `proxy` command keeps its cap opt-in for compatibility.
 
-The local host-pressure gate defaults to holding below 15% available RAM and resuming above 20%
-after two healthy samples; minimum reserves are 1 GiB and 2 GiB, respectively. Tune the percentages
-with `--resource-hold-available-percent` and `--resource-resume-available-percent`; the hold value
-must be lower than the resume value, and neither can exceed 100. Health reports the policy, sampled
-signals, unknown measurements, and reserved memory. These flags do not change Ollama or OS settings.
-
-The reserve applies to every new managed load, not only to the pressure gate: a task is admitted
-when effective available RAM covers already-reserved bytes, the task's estimated bytes, and the
-reserve (the hold reserve normally; the larger resume reserve while capacity is recovering).
-Effective available RAM is free memory plus a bounded reclaimable estimate, so it can be far lower
-than the OS "free" percentage. A refusal reports `assessment.status: "held_insufficient_capacity"`
-with `required_with_reserve_bytes` and `effective_available_bytes`; free memory, choose a smaller
-model, or lower the percentages deliberately.
+The local host-pressure gate reports low available RAM below 15% and clears it above 20% after two
+healthy samples; minimum reserves are 1 GiB and 2 GiB, respectively. Tune the percentages with
+`--resource-hold-available-percent` and `--resource-resume-available-percent`; the hold value must be
+lower than the resume value, and neither can exceed 100. Health reports the policy, sampled signals,
+unknown measurements, and reserved memory. These flags do not change Ollama or OS settings.
+
+On macOS, low available RAM is advisory and never refuses a task. Effective available RAM is free
+memory plus a bounded reclaimable estimate, so it can be far lower than what the OS can actually
+reclaim. When it does not cover already-reserved bytes, the task's estimated bytes, and the reserve,
+FreeLlama first unloads idle unpinned runners, then dispatches anyway: the receipt reports
+`status: "admitted_low_memory_estimate"` and `assessment.memory_advisory: "below_reserve"` with
+`required_with_reserve_bytes` and `effective_available_bytes`. Ollama and the OS decide whether the
+load fits. On Linux, Windows, and other platforms, where available RAM already counts reclaimable
+cache (and is the hard cap in a container), the same shortfall unloads idle runners and then holds the
+new load with `held_insufficient_capacity` until capacity returns, refusing with 503 at its deadline.
+OS memory pressure holds work on every platform. Active swapping and low RAM hold new loads only:
+an already-resident runner adds no memory and stays admitted. CPU-load (one-minute load at or above
+1.5 per logical CPU) and thermal-throttling holds apply to CPU-assigned work and to cold loads, which
+decode weights on the host cores and add a new heat source; a bounded request on a runner already
+resident on the GPU neither waits for the cores nor cools the package by waiting, so it proceeds. The receipt's `resource_admission.assessment.held_by`
+lists the holds that applied to the request.
 Both `serve` and `proxy` accept `--resource-telemetry-policy`:
 
 - `require-memory` (default): wait or refuse local inference when available RAM is unknown, including
@@ -163,7 +168,7 @@ its phase and omits estimates that were not obtained.
 
 See [resource admission](ARCHITECTURE.md#managed-task-execution) for scope and limitations.
 
-`route`, `recommend`, and `task` accept
+`route` and `task` accept
 `--execution-preference auto|prefer-cpu|prefer-gpu`. This is a fallback-capable hint over models
 already assigned by the operator; it never rewrites raw passthrough and never makes an ineligible
 model eligible. Preview the route and inspect the `execution` receipt to confirm whether the hint
@@ -203,13 +208,6 @@ without them it falls back to capability/context-compatible models and reports l
 and missing quality evidence. `quality` requires a policy unless you supply an explicit model.
 An explicit minimum-confidence gate still refuses insufficient evidence in every objective.
 
-`recommend` is side-effect-free. If no installed model qualifies, it can return an installation
-plan from the reviewed recommendation catalog, but it never pulls a model.
-
-`natural-route` asks the configured local intent model to produce structured intent, validates that
-intent, and then invokes the same deterministic router as `route`. It does not let the intent model
-choose the final model directly.
-
 ## Execute tasks
 
 The prompt for `task` is positional:
@@ -233,6 +231,8 @@ Useful task options include:
 - `--priority interactive|normal|background` for fair admission scheduling.
 - `--max-wait-seconds` for a shorter admission/resource wait budget.
 - `--timeout-seconds` for a total deadline including discovery, waiting, loading, and inference.
+- `--stop-on-repetition` to stream the task and stop once the output repeats itself, keeping all
+  generated text.
 - `--defer` to return a job ID immediately.
 
 Inspect deferred work with `jobs`, retrieve a result with `jobs --id <uuid>`, and cancel one task
@@ -266,7 +266,7 @@ benchmark record for the selected model.
 
 ```mermaid
 flowchart LR
-    Q["Quality evaluation aggregate"] --> P["policy-from-eval"]
+    Q["Quality evaluation aggregate"] --> P["freellama policy from-eval"]
     P --> PF["platform.toml"]
     T["Local throughput benchmark"] --> BR["benchmark-report.json"]
     PF --> S["freellama serve"]
@@ -274,36 +274,64 @@ flowchart LR
     S --> C["Medium confidence for configured, measured routes"]
 ```
 
-Generate the policy from correctness data and the benchmark report from local runtime data:
+Generate the policy from correctness data with `policy from-eval` and the benchmark report from
+local runtime data:
 
 ```bash
-npx @octocodeai/freellama policy-from-eval \
-  --aggregate benchmark/local/results/MODEL/aggregate.json \
+npx @octocodeai/freellama policy from-eval \
+  --aggregate aggregate.json \
   --task coding \
   --min-pass 0.8 \
   --out platform.toml
 
 npx @octocodeai/freellama bench-all --output benchmark-report.json
-npx @octocodeai/freellama serve --recommendation-catalog recommendations.example.toml
+npx @octocodeai/freellama serve
 ```
 
-`serve` discovers `platform.toml` and `benchmark-report.json` in its working directory. Explicit
-`--policy-file` and `--benchmark-report` values take precedence.
+`serve` resolves each file in this order: the explicit flag, its environment variable, the working
+directory, then the per-user config directory (`$XDG_CONFIG_HOME/freellama/` or
+`~/.config/freellama/`; `%APPDATA%\FreeLlama\` on Windows).
 
-Replace `MODEL` with the directory containing your completed quality-evaluation aggregate.
+| File | Flag | Environment variable | Discovered name |
+|---|---|---|---|
+| Task policy | `--policy-file` | `FREELLAMA_POLICY_FILE` | `platform.toml` |
+| Benchmark report | `--benchmark-report` | `FREELLAMA_BENCHMARK_REPORT` | `benchmark-report.json` |
 
-`policy-from-eval` reads `pass_at_1`, not the throughput produced by `bench-all`. It refuses expired
-aggregates and fewer than three trials unless `--allow-smoke` is explicit, and it skips models that
-are not installed. This prevents speed data from being mislabeled as quality evidence.
+A task policy file has this shape:
 
-## Compare the CLI and MCP surfaces
+```toml
+schema_version = 1
 
-```bash
-npx @octocodeai/freellama tools
+[policies.code_repair]
+qualified_models = ["qwen3.8:27b-mlx"]   # best first; quality-qualified only
+
+# Optional request defaults for one exact tag on this task: think and num_predict (>= 1).
+[policies.code_repair.model_options."qwen3.8:27b-mlx"]
+think = false
+num_predict = 512
 ```
 
-The command prints the maintained parity map. MCP-only operations are `delegate_research` and the
-online `models { view: "library" }` view. CLI-only operations include `serve`, `proxy`,
-`recommend`, `natural-route`, `bench-all`, `policy-from-eval`, `run`, and `eval`.
+`serve` rejects unknown fields, including `num_ctx`: context size is routing-owned, so set
+`context_tokens` per request instead. Caller request options win over `model_options`, and
+`think = true` applies only to models with the thinking capability. See the annotated
+[platform.example.toml](https://github.com/bgauryy/FreeLlama/blob/main/platform.example.toml).
+
+The MCP server autostarts `serve` with its own environment and an arbitrary working directory, so
+put a machine-wide policy in the config directory or set the variables in the MCP client's `env`.
+`serve` prints each file it picked up.
+
+Replace `MODEL` with the directory containing your completed quality-evaluation aggregate.
+
+`policy from-eval` reads `pass_at_1`, not the throughput produced by `bench-all`. It refuses
+expired aggregates and fewer than three trials unless `--allow-smoke` is explicit, and it skips
+models that are not installed on `--ollama-endpoint` (default `FREELLAMA_OLLAMA_ENDPOINT`, then
+`http://127.0.0.1:11434`). This prevents speed data from being mislabeled as quality evidence.
+It accepts the task kinds `route` accepts, with `-` or `_` (`code-repair` or `code_repair`), writes
+to stdout unless `--out` is given, and reports a refusal as one `error: ...` line with exit status 1.
+
+## Compare the CLI and MCP surfaces
+
+MCP-only operations are `delegate_research` and the online `models { view: "library" }` view.
+CLI-only operations include `serve`, `proxy`, `bench-all`, `policy from-eval`, and `auth-token`.
 
-For the MCP tool contracts, read the [MCP package reference](../packages/mcp/README.md).
+For the MCP tool contracts, read the [MCP package reference](https://github.com/bgauryy/FreeLlama/blob/main/packages/mcp/README.md).

package/docs/CPU_GPU_ROUTING.md

@@ -9,7 +9,7 @@ memory checks, and model transitions for each backend; the operator configures t
 eligible tags. A configured CPU backend is intent: inspect the placement observation before
 treating it as proof of CPU execution.
 
-This configuration is useful for embeddings, intent interpretation, and other small helper models.
+This configuration is useful for embeddings, short classification, and other small helper models.
 Benchmark large CPU workloads before adopting them. Apple silicon shares memory bandwidth; systems
 with discrete GPUs have different contention and transfer costs, so neither result transfers
 without measurement.
@@ -28,7 +28,9 @@ machine-specific:
 
 `doctor` reports `machine.memory_bytes` on macOS, Linux, and Windows. It reports
 `machine.unified_memory_bytes` only when unified memory is known; that compatibility field is not a
-VRAM estimate on discrete-GPU hosts. Model-library fit is therefore a host-memory preflight, and
+VRAM estimate on discrete-GPU hosts. `machine.gpu` names the accelerator when the OS reports it
+(Apple model and core count, or NVIDIA name and memory on Linux), and `performance_cpus` /
+`efficiency_cpus` split hybrid CPUs on macOS. These are identity facts, not placement proof. Model-library fit is therefore a host-memory preflight, and
 actual accelerator placement must come from Ollama's resident-runner report.
 
 An agent can choose `auto`, `prefer_cpu`, or `prefer_gpu`, but it cannot invent topology. The
@@ -352,7 +354,6 @@ and `execution.lifecycle`; a failed unload is reported without discarding the co
 Prefer the following assignments:
 
 - Small embedding models
-- Lightweight intent models
 - Short extraction or classification helpers with measured CPU behavior
 
 Benchmark these assignments before adopting them: