@octocodeai/freellama-mcp-server 0.10.0 → 0.12.0

package/docs/CPU_GPU_ROUTING.md

@@ -4,6 +4,10 @@ To separate GPU work from CPU helpers, configure two Ollama processes and assign
 tags to the secondary process. FreeLlama routes managed tasks within those assignments, applies
 `num_gpu: 0` to CPU requests, and reports both the selected backend and observed processor placement.
 
+This guide describes the Ollama backend contract. The [llama.cpp adapter](LLAMACPP.md) shares
+CPU/GPU admission pools by operator assignment, but does not inject Ollama controls, observe
+physical placement, manage lifecycle, or participate in automatic placement learning.
+
 The calling agent submits tasks and placement preferences. FreeLlama coordinates admission,
 memory checks, and model transitions for each backend; the operator configures the processes and
 eligible tags. A configured CPU backend is intent: inspect the placement observation before
@@ -56,8 +60,10 @@ flowchart TB
     CP --> CPU["CPU runner"]
 ```
 
-FreeLlama does not rewrite raw passthrough request bodies or route them by model. A direct Ollama client that
-needs model-aware CPU placement must use the managed task endpoint instead.
+Native Ollama passthrough does not apply `--cpu-model` assignments; a direct Ollama client that
+needs model-aware CPU placement must use managed tasks. Configured OpenAI-compatible `/v1/*`
+upstreams are a separate model-routing path, and `/v1/systemone` has an asset-translation exception.
+See [API passthrough](API.md#ollama-compatible-passthrough).
 
 ## Start the backends
 
@@ -150,6 +156,9 @@ The defaults above admit one ordinary primary-backend chat (cost 2) and one CPU
 at the same time. They are conservative workload units for the common `OLLAMA_NUM_PARALLEL=1`
 layout, not a profile of the development Mac. Increase them only after measuring queue wait,
 resident memory, and Ollama's own parallel setting on the target host.
+A task waiting for host memory releases execution units while retaining one bounded queue
+registration; returning to admission transfers that ticket atomically. Cancellation removes it.
+This protects execution capacity without creating an uncounted memory-wait backlog.
 Queue cardinality and wait time are separate bounds: excess waiters are refused immediately, while
 an admitted waiter may remain until `--max-queue-wait-seconds`. Automatic backend selection checks
 that the actual task cost fits the current slot snapshot before following historical speed.
@@ -274,7 +283,7 @@ the entire pool's ceiling. Default vision weight four can serialize two vision r
 four-unit budget even when Ollama has two slots. A lower weight requires a qualified workload;
 it does not add engine slots or relax memory checks. See
 [monitoring controls](MONITORING.md#adaptive-concurrency).
-Raw generation requests follow the same rules on the primary backend. A raw request for a runner
+Raw Ollama generation requests follow the same rules on the primary backend. A raw request for a runner
 Ollama already holds at a matching digest and context takes no lock. A raw load takes the load lock
 and receives 503 (`proxy busy: another model load is active`) if it stays busy past the raw queue
 wait. Metadata GET/HEAD, `/api/show`, model-store calls (pull, push, copy, delete), and empty
@@ -312,9 +321,10 @@ The rules that follow:
 - Observing runner readiness wakes resource waiters to re-size their demand. A same-model follower
   waiting for a cold-load forecast can then share the runner while its first request still decodes.
 
-A cold task reads `/api/ps` six times in all: discovery, the receipt's capacity preview, the
-dispatch-time forecast, the observation before the lock, the observation after winning it, and the
-placement observation after the run. A resident task skips the post-lock read.
+Ollama residency is observed during discovery, capacity planning, dispatch preparation and
+post-run placement checks. The cold path rechecks after acquiring the load lock; a resident task
+skips that read. Revalidation and readiness polling can add observations, so the count is not a
+fixed per-task API contract.
 
 ### Degraded runner identity
 
@@ -347,7 +357,7 @@ alike (`PlacementRule`). Placement is judged from the same `/api/ps` entry's `si
 
 ### Raw generation whose allocation cannot be forecast
 
-A raw generation is admitted only with a memory forecast. When the model's metadata cannot be read
+A raw Ollama generation is admitted only with a memory forecast. When the model's metadata cannot be read
 in time, or the request body names a model the proxy cannot resolve (a multipart body, say) and
 could still load one, nothing could size or serialize that load. It is refused with HTTP 503,
 `Retry-After: 1` and `code: "model_forecast_unavailable"` instead of being admitted with a zero-byte
@@ -391,7 +401,8 @@ cold load: when the faster backend's model is not loaded but the other backend's
 the loaded one runs (`resident_backend_avoids_cold_load`); warm the faster model to switch. The CLI
 keeps the bounded, versioned, prompt-free snapshot in the shared database by default; `--ephemeral`
 opts into reset-on-restart behavior. Aggregate duration counters are diagnostic only and do not
-qualify learned hints; a snapshot of another schema version is refused. Health's
+qualify learned hints. A corrupt or unsupported feedback snapshot is rejected with a warning;
+the server starts with empty feedback and preserves the rejected snapshot. Health's
 `decision_ready` describes the stored profile window; the current request must independently
 match it before routing can use its speed. `GET /_freellama/v1/health` exposes profile identity,
 profile sample counts, persistence status, and per-backend admission capacity. Task receipts include
@@ -483,7 +494,9 @@ If an assigned model does not appear, check the following conditions:
 
 A configured but unreachable CPU backend causes model discovery to fail closed. GPU-managed tasks
 can also be unavailable until the secondary server returns. Raw passthrough to the primary server
-continues to work because it does not depend on catalog discovery.
+continues to work because it does not depend on catalog discovery. Explicit configured llama.cpp
+previews and tasks also bypass Ollama catalog discovery; use their route preview rather than the
+Ollama-oriented overall readiness endpoint.
 
 For the surrounding Ollama settings, see [Production](PRODUCTION.md#configure-ollama-explicitly). For
 all `serve` flags, see the [CLI reference](CLI.md).

package/docs/DEFERRED_JOBS.md

@@ -1,6 +1,6 @@
 # Deferred jobs and recovery
 
-FreeLlama's deferred jobs keep one accepted task identity through polling, cancellation and result retrieval. A missing result or an uncertain outcome never authorizes automatic inference replay. Ollama still owns execution and physical cancellation; these records coordinate the FreeLlama worker.
+FreeLlama's deferred jobs keep one accepted task identity through polling, cancellation and result retrieval. A missing result or an uncertain outcome never authorizes automatic inference replay. The configured inference server owns execution and physical cancellation; these records coordinate the FreeLlama worker. The [llama.cpp adapter](LLAMACPP.md) supports only its documented bounded text subset, even when deferred.
 
 ## Enable durable receipts
 
@@ -14,9 +14,9 @@ The journal persists receipts, bounded results/errors, and keyed hashes of reque
 
 `POST /_freellama/v1/tasks` with `defer:true` returns `deferred:true`, a `job` containing its `id`, and `retention` limits. Use `GET /_freellama/v1/jobs/{id}` for its result and `GET /_freellama/v1/jobs` for a metadata-only list. The list strips result/error payloads in SQLite before loading recovered records into the control plane.
 
-An embedding task may carry `nearest` (`{query?, top?, min_score?, labels?}`, with `return_embeddings` to keep the vectors). The core embeds the items (the query first, in slices of 32 pinned to one verified runner identity), ranks them by cosine similarity and stores `nearest: {mode, matches|pairs}` in place of vectors, so a deferred job's result is ranked and small. The whole input is one job and must fit the 16 MiB job input limit; without a query at most 4,000 items are compared pairwise. A malformed `nearest` is refused at submission. Through MCP, `run_task {defer:true, tasks:[...]}` submits every item as its own job and returns `{deferred:true, jobs:[{id, job}]}`; the HTTP `/tasks/batch` endpoint still refuses deferred items.
+An embedding task may carry `nearest` (`{query?, top?, min_score?, labels?}`, with `return_embeddings` to keep the vectors). The core embeds the items (the query first, in slices of 32 pinned to one verified runner identity), ranks them by cosine similarity and stores `nearest: {mode, matches|pairs}` in place of vectors, so a deferred job's result is ranked and small. The whole input is one job and must fit the 16 MiB job input limit; without a query at most 4,000 items are compared pairwise. A malformed `nearest` is refused at submission. Through MCP, `run_task {defer:true, tasks:[...]}` submits every item as its own job and returns `{deferred:true, jobs:[{id, job}]}`; the HTTP `/task-batches` endpoint still refuses deferred items.
 
-`POST /_freellama/v1/jobs/{id}/cancel` requests cancellation and waits for the attached worker to settle and release local permits. It also waits for pending terminal persistence. A result already committed by the execution path retains its completed status. Acknowledged local cancellation is not proof that Ollama physically stopped accepted work.
+`POST /_freellama/v1/jobs/{id}/cancel` requests cancellation and waits for the attached worker to settle and release local permits. It also waits for pending terminal persistence. A result already committed by the execution path retains its completed status. Acknowledged local cancellation is not proof that the inference server physically stopped accepted work.
 
 `DELETE /_freellama/v1/jobs/{id}` first settles locally owned work, then hides the record and removes its retained payload. With persistence enabled, its idempotency tombstone remains until expiry. An unattached worker with an unknown outcome cannot be removed early. A failed terminal database commit also prevents removal of its still-unfinished durable record.
 
@@ -26,10 +26,23 @@ MCP task cancellation and managed research use addressable jobs regardless of wh
 
 ## Idempotency
 
-With persistence enabled, the HTTP task request may include `idempotency_key`, containing 1–128 UTF-8 bytes. It requires `defer:true`; synchronous requests and servers without durable jobs reject the field. The current MCP tool schema does not add an idempotency-key argument.
+With persistence enabled, the HTTP task request may include `idempotency_key`, containing 1–128 UTF-8 bytes. It requires `defer:true`; synchronous requests and servers without durable jobs reject the field. MCP passes it as `run_task {defer:true, idempotencyKey}` and the agent commands as `freellama run --defer --idempotency-key KEY`; both refuse a key without `defer`. A batch (`tasks`, `--each`) takes no key: each item is its own job.
 
 Keys are hashed (HMAC-SHA256, see [request hashes](#request-hashes)) with authenticated caller identity and scoped to a server topology: listen address, primary endpoint, CPU endpoint and configured CPU model list. Keep that configuration and caller identity stable when recovering jobs. Task input is serialized and hashed after removing the key, together with the set of explicitly supplied fields. Retries must preserve both values and field presence: omission can inherit scope defaults while an explicit default overrides them. Reusing a key with changed input returns `task_job_idempotency_conflict`.
 
+Managed llama.cpp configuration participates in the **input hash**, while the lookup namespace
+stays stable. The fingerprint covers each managed engine's name, URL, model, processor and
+context/output/memory bounds, sorted by name. Declaration order, API-key rotation and passthrough-only
+upstreams do not change it. Adding, removing or changing a managed mapping returns a conflict for a
+retained key instead of returning a result from different settings or silently running new work.
+The old job remains readable by ID. All managed mappings are included conservatively because a
+scope can inherit a model: even changing an unrelated managed engine can invalidate a retry.
+
+Ollama-only installations preserve their existing input hashes. After upgrading an installation
+with managed engines from a version without this fingerprint, retained keys conflict; retrieve
+and inspect the original job first. Use a new key only for intentionally new work. The fingerprint
+describes operator configuration, not a llama.cpp weight digest or physical execution attestation.
+
 A repeated key returns the original job, including a retained terminal result, before checking local registry capacity. Claiming the key and reserving its durable record is one transaction. Concurrent requests cannot both claim that key. A new durable claim rejected by local capacity is recorded as failed with `execution_outcome:"not_started"` and `retry_safe:true`; repeating the key returns that original refusal instead of starting work.
 
 After restart, only recorded terminal outcomes are recoverable as settled results. An unfinished record becomes a read-time `failed` receipt with reason `execution_outcome_unknown`, error code `task_job_outcome_unknown`, its original `job_id`, and `retry_safe:false`. It may represent an interrupted worker, a worker still attached to another server, or a submission interrupted around acceptance. No worker is resumed or recreated. If the original worker later commits a terminal record, subsequent retrieval can observe it.

package/docs/FEATURES.md

@@ -1,15 +1,16 @@
 # FreeLlama features and how it works
 
-Moved from the root README. For install and a first task, start with the [README](../README.md).
+For install and a first task, start with the [README](../README.md). The
+[end-to-end review](OLLAMA_REVIEW.md) records feature coverage and current limits.
 
 ![A brown cartoon llama stands on a pastel cloud background.](../assets/logo.jpg)
 
 **Run your agent's bulk work on local models without overloading your machine.**
 
-FreeLlama sits between an AI agent and [Ollama](https://ollama.com). For each task it picks an
-installed model that fits, admits the task against live memory, CPU and heat signals, runs it on the
-GPU or a CPU-only Ollama, and returns the result with a receipt of where and how it ran. Any Ollama
-model works.
+FreeLlama sits between an AI agent and local inference servers. Its full Ollama path selects an
+eligible installed model, checks memory and pressure, and returns a result with execution evidence.
+An explicit [llama.cpp adapter](LLAMACPP.md) supports bounded text tasks with the same admission
+and memory governor. Capabilities, context and backend support determine which tasks can run.
 
 ```mermaid
 flowchart LR
@@ -22,8 +23,8 @@ flowchart LR
 ## Why FreeLlama
 
 Coding agents spend paid tokens and context on work a local model can do: embedding files, reading
-screenshots and scans, rewriting or summarizing many files. Calling Ollama directly, an agent cannot
-tell:
+screenshots and scans, rewriting or summarizing many files. A direct Ollama caller must assemble
+metadata, host observations and result verification to answer:
 
 - **Which model:** tag names do not show capabilities, context window or memory footprint.
 - **Whether there is room:** a new load can push the machine into swap, several agents can pile onto
@@ -55,8 +56,9 @@ flowchart TD
 
 ### The smart flow
 
-Every managed task passes the same gates, each deciding from installed metadata, live host signals
-and observed behavior, never from a model's name.
+Every managed task passes core admission and resource checks. The Ollama path below selects from
+installed metadata and qualified evidence. llama.cpp uses explicit operator configuration and
+refuses capabilities outside its [documented subset](LLAMACPP.md#supported-contract-and-limits).
 
 ```mermaid
 flowchart LR
@@ -76,29 +78,35 @@ An explicit `model` always wins over routing.
 ## Features
 
 Each feature is reachable from MCP, the CLI and the HTTP control API under `/_freellama/v1` unless a
-cell says otherwise.
+cell says otherwise. Every MCP tool is also reachable from a shell with `freellama tool <name> JSON`, and
+`run_task` with `freellama run` ([agent CLI](AGENT_CLI.md)).
 
 | Feature | What you get | MCP tool | CLI | HTTP |
 |---|---|---|---|---|
-| [Model selection](#model-selection) | The right installed model for the task, with a confidence level | `run_task {preview:true}`, `models` | `route`, `models` | `POST /routes`, `GET /models` |
+| [Model selection](#model-selection) | An eligible installed model with an evidence grade | `run_task {preview:true}`, `models` | `route`, `models` | `POST /routes`, `GET /models` |
+| [Fallback chains](#fallback-chains) | An ordered list of models to try when the one before is refused before it runs (no room, input too long, not installed); never after a model started | `run_task {fallbackModels}` | `task --fallback-model`, `freellama run --fallback` | `fallback_models` on `POST /tasks` |
 | [Resource admission](#resource-admission) | Tasks run, wait or are refused with a reason | every `run_task` receipt, `doctor {view:"status"}` | `status`, `ready` | `GET /health` (liveness), `GET /ready` (can serve now), `GET /status` |
 | [CPU and GPU backends](#cpu-and-gpu-backends) | Helper models on CPU next to GPU work, with proof of where each ran | `run_task {executionPreference}` | `serve --cpu-upstream --cpu-model` | `POST /tasks` |
 | Decisions | Label, triage or yes/no-check items with a decision model: one forward pass, one line per answer | `run_task {task:"decision", state, questions}` | `task --task decision --questions` | `POST /tasks` |
 | Ranked search and duplicates | Embed files or texts and get the closest items to a query, or the most similar pairs, as labels with scores instead of vectors; files are chunked by lines and labelled `path:Lstart-Lend` | `run_task {task:"embedding", files, nearest}` | `task --task embedding --input-file F --nearest QUERY` | — |
-| Speech | Transcribe or translate a WAV, or speak text to a WAV file, on a local audio model | `audio` | — | Ollama `/v1/audio/*` through serve |
+| Speech | Transcribe or translate a WAV, or speak text to a WAV file, on a local audio model | `audio` | `freellama tool audio` | Ollama `/v1/audio/*` through serve |
 | Inputs by path | The server reads the files an agent names, so logs, screenshots and source files are never pasted into its output | `run_task {files}` | `task --input-file`, `--image` | — |
-| [Loop control](#loop-control) | A repeating answer stops early with every token kept | `run_task {stopOnRepetition}` | `task --stop-on-repetition` | `stop_on_repetition` |
+| [Loop control](#loop-control) | Detect repeated output and stop a known looper; the retained result records the stop reason | `run_task {stopOnRepetition}` | `task --stop-on-repetition` | `stop_on_repetition` |
 | [Machine awareness](#machine-awareness) | GPU, cores, memory, GPU load and heat read from the OS | `doctor` | `machine`, `doctor` | `GET /machine` |
 | [Batches, jobs and history](#batches-jobs-and-history) | Fan-out, submit now and collect later, multi-turn chat without resending history | `run_task {tasks}` / `{defer}`, `task_jobs`, `scope`, `session` | `task --defer`, `jobs`, `scope`, `session` | `/task-batches`, `/jobs`, `/scopes`, `/sessions` |
 | Warm-up and unload | Load a model ahead of use through the same memory checks; unload a resident one once its tasks finish (never loads first) | `ollama_manage {action:"warm"}`, `ollama_manage {action:"unload"}` | `warm`, `unload` | `POST /warm`, `POST /unload` |
 | Model catalog and cookbooks | Tested picks per job with their evidence, what is installed and fits now, and a recipe for each | `models {view:"catalog"}`, `freellama://docs/cookbooks/<job>` | `catalog [--job JOB]`, [docs/cookbooks](cookbooks/README.md) | — |
-| Find and install models | ollama.com and Hugging Face GGUF search, every tag with its size and memory fit, pulls only on approval | `models {view:"library"}`, `ollama_pull`, `ollama_delete` | MCP only | MCP only |
-| Job mode and prompts | Name a job and FreeLlama runs its first catalog pick that is installed and fits, with its documented prompt, options and embedding prefixes; your fields win. MCP prompts give each job's call, and `saveAs` hands an answer to the next task as a file | `run_task {job, saveAs}`, MCP prompts, [cookbook](cookbooks/jobs.md) | MCP only | MCP only |
-| Chunked sweeps | A local model checks a code tree, log or docs site chunk by chunk in the background, saves every result, re-checks flags with a stronger model and, against a baseline, reports only what changed | `sweep`, [cookbook](cookbooks/sweep.md) | MCP only | MCP only |
-| [Grounded research](#grounded-research) | Read-only repository answers with file and line citations | `delegate_research` | MCP only | MCP only |
+| Find and install models | ollama.com and Hugging Face GGUF search, every tag with its size and memory fit, pulls only on approval | `models {view:"library"}`, `ollama_pull`, `ollama_delete` | `freellama tool models\|ollama_pull\|ollama_delete` (same approvals) | — |
+| Job mode and prompts | Name a job: use a qualified machine-local Rust route when available, otherwise a portable catalog/capability fallback, with documented prompts and options; your fields win. MCP prompts give each job's call, and `saveAs` hands an answer to the next task as a file | `run_task {job, saveAs}`, MCP prompts, [cookbook](cookbooks/jobs.md) | `freellama run -j JOB\|-w WORKFLOW --save-as NAME`, `freellama workflows` | — |
+| [Your own workflows](PROMPTS.md#your-own-workflows) | A Markdown file per workflow in `~/.octocode/freellama/workflows/`: a job, a title and the instructions. It runs like a built-in, replaces a built-in of the same name, and the receipt names the file and its digest | `run_task {workflow}` | `freellama run -w NAME`, `freellama workflows` | gateway `freellama/NAME` |
+| Chunked sweeps | A local model checks a code tree, log or docs site chunk by chunk in the background, saves every result, re-checks flags with a stronger model and, against a baseline, reports only what changed | `sweep`, [cookbook](cookbooks/sweep.md) | `freellama tool sweep` | — |
+| [Grounded research](#grounded-research) | Read-only repository answers with file and line citations | `delegate_research` | `freellama tool delegate_research` | — |
 | [Small agent context](FEATURES.md#small-agent-context) | Compact results and paged details instead of raw dumps | every tool, `read_response` | JSON on stdout | JSON |
 | [Observe and evaluate](#observe-and-evaluate) | Live status, usage, metrics, benchmarks, routing policy from evaluations | `doctor {view:"usage"}` | `status`, `usage`, `config`, `bench-all`, `policy from-eval` | `/status`, `/usage`, `/metrics`, `/config`, `/_freellama/ui` |
-| Ollama passthrough | Any Ollama `/api/*` or `/v1/*` call, byte-preserving, with retries and a stream cap | — | `serve`, `proxy` | Ollama's own paths |
+| Ollama passthrough | Any Ollama `/api/*` or `/v1/*` call, byte-preserving, with retries and a stream cap; `freellama gateway` also serves `freellama/<workflow\|job\|auto>` virtual models over Ollama, OpenAI `/v1/chat/completions` and `/v1/responses`, and Anthropic `/v1/messages` | — | `serve`, `proxy`, `freellama gateway` | Ollama's own paths |
+
+Features involving model lifecycle, tool calls, media, embeddings, streaming and automatic
+selection below describe Ollama. The llama.cpp adapter supports explicit text tasks only.
 
 ### Model selection
 
@@ -107,20 +115,59 @@ window, routing policy and memory fit, and reports a confidence level. `minConfi
 refuses weakly evidenced routes, `objective:"quality"` requires a policy-backed model, and `model`
 pins one. A bare name resolves as in Ollama: `name:latest`, or the only installed tag of that family.
 
-Routing never ranks by a model's name or architecture and adds no model-specific request options. A
+Rust routing never ranks by a model's name or architecture. Operator policy can supply per-model
+request defaults; explicit caller options take precedence. Shipped MCP job prompts and portable
+catalog picks are a separate opt-in layer. A
 model that loops on a task kind (at least two loops in at most twice as many runs) ranks last for that
 kind; `models` lists the evidence as `observed_loops`. See [model selection](MODEL_SELECTION.md).
 
+Machine-local throughput rankings require matching host/hardware identity, loopback endpoint and
+model digest; legacy or mismatched reports do not qualify. Throughput is not correctness evidence.
+See [model selection](MODEL_SELECTION.md) and [reliability evidence](RELIABILITY.md).
+
+### Fallback chains
+
+`fallbackModels` (HTTP `fallback_models`, CLI `--fallback-model`, agent CLI `--fallback`) is an
+explicit, ordered list of up to 4 models. The next one runs only when the one before it was refused
+before anything executed:
+
+| Refusal | Code |
+|---|---|
+| No room in memory by the wait deadline | `resource_admission_unavailable` |
+| The input exceeds the model's context window | `context_window_exceeded`, `embedding_item_exceeds_context` |
+| The model file alone exceeds the host budget | `model_exceeds_host_budget` |
+| Not installed, or lacks the task's capability | `model_not_installed`, `model_not_eligible` |
+| The backend's circuit is open | `upstream_circuit_open` |
+
+Anything else stops the chain: a timeout, a cancellation, a busy queue or an error after the model
+started never moves to another model, so no work is replayed after an uncertain outcome. The receipt's
+`fallback` lists every attempt with its code, and the answer gets a `check:` line naming the model
+that answered. Each attempt waits as a single task would, so set a short `maxWaitSeconds` to move on
+sooner; `timeoutSeconds` bounds the whole chain. While one model is tried, the models after it count
+as busy, so making room for an attempt that is then refused never unloads the model that would run
+next. An entry equal to the first model (which may be a scope default, a job's pick or the model
+routing chose) is skipped, and every attempt names the model it really used; a model listed twice
+is refused. A chain that fails is charged in usage to the model that failed last. When an embedding task runs in slices and a later slice is refused,
+part of the work already ran, so the chain stops there. A chain cannot be combined with a pinned
+digest or backend, and MCP refuses one over separately sliced embeddings so one answer comes from one
+model. Batch items and `task_jobs` results carry the receipt's `fallback` too.
+
+```jsonc
+{ "tool": "run_task", "arguments": { "model": "lfm3:1.7b", "fallbackModels": ["bonsai-2:9b"], "files": ["big.log"], "prompt": "summarize", "maxWaitSeconds": 20 } }
+```
+
 ### Resource admission
 
-A task queues fairly, waits for budget units on its backend and for the host signals that apply to
-it, then runs or is refused with `Retry-After`.
+A task queues fairly, waits for budget units on its assigned pool and for applicable host signals,
+then runs or is refused. Resource waiters release execution units and retain bounded queue tickets;
+cancellation returns those tickets and reservations. Busy and resource refusals carry retry guidance.
 
 - **Budget.** Each Ollama decoding slot (`OLLAMA_NUM_PARALLEL`) is worth two units: a chat costs 2, a
   vision task 4 (capped to the budget), an embedding 1 per four inputs. The GPU queue holds 16 tasks,
   the CPU queue 8. A task waits at most 120 s, or `maxWaitSeconds`.
-- **Fairness.** Priority classes get 3, 2 and 1 turns per round, FIFO within a class. After six
-  bypasses the oldest blocked task reserves capacity, so nothing starves. In a live run, 4
+- **Fairness.** Priority classes get 3, 2 and 1 turns per round. Bounded model affinity can reorder
+  a class before aging restores FIFO. After six capacity bypasses the oldest blocked task reserves
+  released capacity, preventing indefinite bypass by smaller tasks. In a live run, 4
   interactive tasks submitted last finished before the normal and background tasks queued ahead.
 - **Fast refusals.** A full queue answers 429 at once, an expired wait 503. Both carry `Retry-After`
   from the tasks ahead, how many of that cost run at once, and recent service time.
@@ -132,32 +179,32 @@ Each host signal holds only the work it affects:
 | OS memory pressure (macOS `kern.memorystatus_vm_pressure_level`, Linux PSI) at warning | new model loads; at critical, everything |
 | Low available RAM | judged per load: a new load runs if it leaves its own reserve, else waits; a loaded model keeps serving |
 | Active swapping | new loads of 1 GiB or more, or of unknown size; a loaded model keeps serving, raw passthrough included |
-
-A load that waits for memory is told who holds it: the 503, the progress message and the route
-preview name the largest programs (`node: tsserver.js ×3 3.8 GB, rust-analyzer ×2 2.3 GB`).
-| CPU load | CPU-bound work only, never a GPU load or GPU inference |
+| CPU load | CPU-assigned work; all work when no accelerator is observed |
 | Thermal throttling (macOS) | CPU-bound work and new loads |
 
+An Ollama load waiting for memory can report the largest observed memory holders in its refusal,
+progress and preview (for example, `node: tsserver.js ×3 3.8 GB`). Missing observations stay unknown.
+
 A new load must leave a reserve free: 15% of RAM, at least 1 GiB, capped at the load's own size. If
-it does not fit, idle models are unloaded first; if it still does not fit, it waits and is refused at
+an Ollama load does not fit, eligible idle Ollama models are unloaded first; if it still does not fit, it waits and is refused at
 its deadline, on every OS. On macOS the available figure already includes reclaimable pages
 (inactive, speculative, purgeable), so loading past it means compression and swap.
 `serve --low-memory advisory` (or `FREELLAMA_LOW_MEMORY=advisory`) restores admit-with-warning.
 While any model is loaded, `serve` also samples the host itself: under low memory, OS pressure or
 swapping it unloads the cheapest idle models (audit trigger `pressure_sweep`), never pinned,
-`keep_alive: -1` or busy ones. Because of the reserve, a load can wait while the OS still shows free
+`keep_alive: -1` or busy ones, nor one another client used in the last 15 s. Because of the reserve, a load can wait while the OS still shows free
 memory ([admission controls](CLI.md#bound-admission)). Total deadlines (`timeoutSeconds`) and
 per-backend circuit breakers complete admission; see [monitoring and tuning](MONITORING.md).
 
 ### Model memory and switching
 
-A loaded model answers at once, a cold one first pays its load time, and loading past what the
-machine holds swaps and stalls everything. FreeLlama decides what stays loaded:
+On the Ollama path, compatible residency avoids a cold load but still requires admission and
+applicable pressure checks. FreeLlama coordinates which Ollama runners stay loaded:
 
 ```mermaid
 flowchart TD
     Req(["Request names a model<br/>managed task or raw Ollama call"]) --> Res{"Loaded at a<br/>compatible context?"}
-    Res -->|yes| Serve(["Serve now<br/>memory holds never apply"])
+    Res -->|yes| Serve(["Reuse runner<br/>admission and critical pressure still apply"])
     Res -->|no| Size["Size the load<br/>Ollama's measured footprint, else file + KV + graph"]
     Size --> Room{"Room needed?"}
     Room -->|"GPU VRAM short"| Plan
@@ -182,9 +229,9 @@ flowchart TD
 - **Audit.** `status` shows the budget in use, loads, unloads and evictions by cause, switch time per
   model, and models swapped back and forth. Every MCP result carries `timing` (wall time, plus queue,
   load, prompt and output time for a task).
-- **Learned per model.** Each model revision's output needs (reasoning included), load time, memory
-  footprint and loops are measured from its runs and saved. FreeLlama sends no output cap of its own;
-  a model that ran out of room gets twice as much next time.
+- **Learned per model.** Ollama model-revision output needs, load time and memory footprint can be
+  retained from qualified runs. Repetition-loop counters are process-local. Ollama text tasks have
+  no automatic output cap unless the caller or policy sets one; llama.cpp always has an operator cap.
 - **Preload.** `preload.startup` loads listed models at startup; with `messages` (such as a long
   system prompt) it also evaluates them into the runner's prompt cache. Predictive preload counts
   model-to-model switches and, once a successor is clear, loads it in the background, only when it
@@ -192,7 +239,8 @@ flowchart TD
   `ollama_manage {action:"warm", model, systemPrompt}`.
 - **Shared state.** Measured sizes, load times, placement feedback, usage and the audit live in one
   `SQLite` file, `~/.octocode/freellama/freellama.sqlite`, shared by every FreeLlama process on the
-  machine.
+  machine. Admission pools and memory reservations remain process-local; shared SQLite does not
+  coordinate competing servers' execution capacity.
 
 Measured switch times on an Apple M2 Pro (32 GB) through `serve`, model files in the OS file cache:
 
@@ -220,7 +268,8 @@ model, or give helpers their own backend.
   request for `lmstudio/qwen3-8b` goes to upstream `lmstudio` as `qwen3-8b`; a bare name goes to
   Ollama when Ollama has it, else to the engine that lists it. `GET /v1/models` lists all. A local
   engine waits out host holds as a loaded model would. The caller's FreeLlama token never leaves;
-  each engine gets its own key, sent only over HTTPS or to this machine.
+  each engine gets its own key, sent only over HTTPS or to this machine. A local llama.cpp server
+  can opt into [managed text execution](LLAMACPP.md) with shared admission and memory reservations.
 
 ```toml
 # ~/.octocode/freellama/clients.toml
@@ -237,8 +286,8 @@ url = "http://127.0.0.1:1234/v1"
 
 ### CPU and GPU backends
 
-An optional second, CPU-only Ollama runs exact helper models, such as an embedder, so they never
-compete with GPU work: `serve --spawn-cpu-backend --cpu-model TAG` starts it on loopback, waits for it
+An optional second, CPU-configured Ollama runs exact helper models alongside GPU work; both can
+still contend for host memory, bandwidth and power: `serve --spawn-cpu-backend --cpu-model TAG` starts it on loopback, waits for it
 and stops it with `serve`, or point `--cpu-upstream` at one you run. Each backend has its own budget, queue and model-transition lock. The operator
 assigns models; agents can only state an `executionPreference`. Ask for
 `minPlacementEvidence:"observed"` when placement matters, and trust the receipt's observed placement
@@ -249,7 +298,7 @@ over the configuration. See [CPU/GPU routing](CPU_GPU_ROUTING.md).
 Some models answer correctly, then repeat the answer until their output limit (a `num_predict` cap or
 the end of their context). With the repetition stop, FreeLlama streams the answer, checks it every 16
 tokens, and stops once the text has repeated twice over at least 96 characters; closing the
-connection cancels generation in Ollama. It is on by default for free text with no output cap (JSON
+connection requests cancellation in Ollama; local closure is not proof of physical stop. It is on by default for free text with no output cap (JSON
 and tool calls are left whole); `stopOnRepetition:false` (CLI `--stop-on-repetition false`) turns it
 off, `true` turns it on for a capped task. Nothing is trimmed: `response` holds the full text and
 `output_check` says where the first copy ends. A loop that ran to the limit without the stop is
@@ -277,7 +326,10 @@ privileges:
 | Keep related calls on one model | `session {action:"create"}`, then `run_task {sessionId}` |
 | Hot model before latency-sensitive work | `ollama_manage {action:"warm", model, contextTokens}` |
 
-Scopes, sessions and deferred jobs live in memory and are lost when the service restarts. See
+Scopes and sessions live in memory and are lost when the service restarts. Deferred jobs are
+process-local by default; with `FREELLAMA_DURABLE_JOBS=1` their finished receipts survive a restart,
+an unfinished one reads back as an unknown outcome (never replayed), and `idempotencyKey` makes a
+resubmission return the original job ([deferred jobs](DEFERRED_JOBS.md)). See
 [scopes and warming](SCOPES_AND_WARMING.md).
 
 ### Grounded research
@@ -291,7 +343,8 @@ computed verdict. Discard `escalate` results and verify the citations on the res
 `run_task` and `delegate_research` return a compact view unless `receipt:"full"`; every tool saves its
 whole result to a `response_file`, which `read_response` pages by section. Embedding vectors are
 omitted unless `returnEmbeddings:true`. The optional `telemetry.externalEquivalent` estimates avoided
-API cost from a rate card you configure. See [token economics](../packages/mcp/README.md#token-economics).
+API cost from a rate card you configure with the `FREELLAMA_EXTERNAL_COST_*` variables in
+[MCP configuration](../packages/mcp/CONFIGURATION.md#environment-variables).
 
 ### Observe and evaluate
 
@@ -300,4 +353,5 @@ API cost from a rate card you configure. See [token economics](../packages/mcp/R
 - **Evaluation:** `bench-all` benchmarks installed models; `policy from-eval` writes a routing policy
   from quality-evaluation results. See [model selection](MODEL_SELECTION.md).
 - **Ollama compatible:** `/api/*` and `/v1/*` reach Ollama byte-for-byte, streaming included;
-  FreeLlama never rewrites raw bodies. See [Ollama sidecar](ARCHITECTURE.md#ollama-sidecar).
+  configured `/v1/*` upstream routing and `/v1/systemone` asset translation are explicit exceptions
+  to byte-preserving primary passthrough. See [Ollama sidecar](ARCHITECTURE.md#ollama-sidecar).

package/docs/LLAMACPP.md

@@ -0,0 +1,133 @@
+# Managed llama.cpp text tasks
+
+FreeLlama can run bounded text tasks on one local, operator-run `llama-server`. Configure the
+server in `upstreams.toml`, then explicitly select `name/model` through `freellama task` or the
+managed HTTP API. The Rust core owns weighted admission, host memory reservations, total deadlines,
+usage accounting and result normalization. llama.cpp owns model loading and physical execution.
+
+Mock HTTP contracts verify admission, deadlines, refusal paths and result normalization. One live
+managed completion also passed on 2026-11-14 with llama.cpp 0.8.0, build 11960, commit `8642541b2`,
+using an existing 1.1 GB GGUF: preview returned HTTP 200 without inference, then the managed task
+returned `hello` with 17 prompt tokens and 4 output tokens. The primary Ollama endpoint was
+intentionally unreachable. Both temporary processes exited cleanly. See the
+[configuration and raw receipts](assets/llamacpp-managed-smoke-2026-11-14.json).
+
+That single smoke test establishes this basic protocol path on that build. It does not qualify
+throughput, task accuracy, physical placement or the external server's cancellation behavior.
+
+## Configure a server
+
+Start a single-model llama-server using a GGUF file you already have. For example, after checking
+your installed llama.cpp build's help:
+
+```bash
+llama-server --model /absolute/path/model.gguf --alias text-model \
+  --host 127.0.0.1 --port 8080 --ctx-size 4096 --parallel 1 --n-gpu-layers 0
+```
+
+This example selects CPU execution. FreeLlama does not run this command or download a model.
+Keep the server configuration stable while tasks are active. Use a separate endpoint for each
+model; router mode and automatic model switching are outside this adapter's supported contract.
+
+Add this to `upstreams.toml` in the FreeLlama home, or pass its path with `serve --upstreams`:
+
+```toml
+[[upstream]]
+name = "cpp"
+url = "http://127.0.0.1:8080/v1"
+# api_key_file = "keys/llamacpp.key"
+
+[upstream.llamacpp]
+model = "text-model"
+processor = "cpu"
+context_tokens = 4096
+max_output_tokens = 256
+memory_bytes = 4294967296
+```
+
+Choose `memory_bytes` for your actual model and server configuration. The example's 4 GiB is an
+illustration, not a sizing recommendation. It is a conservative reservation for each active
+request, covering a possible model load and runtime buffers. FreeLlama takes no residency credit,
+so an already loaded model can be counted twice. The operator must choose sufficient headroom;
+this reservation does not prove fit or measure llama.cpp allocation. Concurrent requests each
+reserve this amount. Host pressure and memory policy apply through the same governor as Ollama.
+A request blocked on resources releases its execution units after the initial bounded telemetry
+check. It keeps one ticket in the shared bounded queue, then reacquires execution capacity after
+reserving memory. Cancellation releases the ticket and any reservation; a full queue refuses new
+waiting work.
+
+`processor` selects the existing CPU or GPU admission pool and pressure policy. It never changes
+the server's placement. `observed_processor` stays null and physical placement stays unknown.
+`min_placement_evidence:"observed"` refuses the request.
+
+The URL must be loopback HTTP(S), end in `/v1`, and contain no URL credentials, query or fragment.
+An optional key file follows the existing owner-only secret-file rules. FreeLlama sends only that
+upstream key to llama.cpp; it never forwards the caller's FreeLlama token.
+
+## Preview and execute
+
+```bash
+freellama serve --upstreams ./upstreams.toml
+```
+
+In another terminal:
+
+```bash
+freellama route --model cpp/text-model --task completion
+freellama task --model cpp/text-model --task completion "Explain this function in one sentence."
+```
+
+Equivalent HTTP calls to a default local server:
+
+```bash
+curl http://127.0.0.1:11435/_freellama/v1/routes \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"cpp/text-model","task":"completion"}'
+
+curl http://127.0.0.1:11435/_freellama/v1/tasks \
+  -H 'Content-Type: application/json' \
+  -d '{"model":"cpp/text-model","task":"completion","prompt":"Say hello.","timeout_seconds":30,"request_options":{"options":{"num_predict":32}}}'
+```
+
+Add your bearer token if the FreeLlama listener requires authentication. Preview verifies readiness
+and the model listing, returns a capacity snapshot, and reserves nothing. Execution repeats the
+observation, acquires the shared weighted pool and memory reservation, and sends one generation
+POST. Explicit llama.cpp routes and tasks do not require Ollama to be reachable. The general model
+catalog, auto-selection, benchmark qualification and overall `/ready` health remain Ollama-oriented.
+Use the explicit route preview to check this engine.
+
+The result retains `response.message.content`, `response.done_reason`, usage token counts and
+`execution.engine:"llamacpp"`. `finish_reason:"length"` is retained as `done_reason:"length"`;
+callers must treat it as a potentially truncated answer. Repetition is checked after completion.
+Failures and uncertain POST outcomes are never automatically replayed.
+
+Deferred jobs and text-only scopes use the core's [job](DEFERRED_JOBS.md) and
+[history](SCOPES_AND_WARMING.md) contracts. They do not add engine capabilities: retained messages
+must still be text-only and the fixed context restriction still applies. Built-in MCP jobs or
+workflows can add Ollama-specific options such as thinking or structured-output controls, so use a
+plain explicit text task unless every resulting field is supported. Automatic job selection does
+not discover this engine.
+
+## Supported contract and limits
+
+| Surface | Behavior |
+|---|---|
+| Task kinds | `completion`, `coding`, `long_context`; explicit configured model only |
+| Input | Nonempty `prompt`, or text-only `messages` with system/user/assistant roles |
+| Output bound | Positive `max_output_tokens` is mandatory; `num_predict` can lower it |
+| Sampling | `request_options.options.temperature`, `top_p`, `seed`, `stop` |
+| Context | Fixed operator context; an explicit `context_tokens` must match it; exact input token fit is not verified by FreeLlama |
+| Deadlines | Shared task deadline and queue cap; cancellation closes the upstream connection and releases core permits; the external server owns physical cancellation |
+| Response buffering | Discovery is capped at 64 KiB; generation uses `FREELLAMA_MAX_REQUEST_BYTES` as its response cap too |
+| Tool calling, `code_repair`, images, embeddings, decision tasks | Refused before dispatch |
+| Streaming, early repetition stop, format/schema, thinking/logprobs controls | Refused before dispatch; output always has a finite token cap |
+| Warm/unload, model download/delete, identity pins | Unsupported on the managed adapter; server lifecycle remains operator-owned |
+| Placement and confidence | Configured assignment only; observed placement and benchmark confidence requirements refuse |
+| Automatic routing and learning | No auto-selection; llama.cpp results do not train Ollama preload/residency/placement facts |
+
+An upstream without `[upstream.llamacpp]` remains OpenAI-compatible passthrough only. Raw `/v1/*`
+requests still use that passthrough contract and do not inherit this adapter's managed task limits.
+
+The wire behavior follows the [llama.cpp server reference](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md):
+`/v1/health`, `/v1/models`, and nonstreaming `/v1/chat/completions`. The adapter requires a ready
+server listing exactly one model with the configured ID; a reported model status must be `loaded`.