@octocodeai/freellama-mcp-server 0.10.0 → 0.12.0

package/docs/LOCAL_MODELS.md

@@ -1,9 +1,10 @@
 # Local model catalog
 
-Which local model to use for each job on one machine, what was measured, and what is only
-published. The method for choosing a model on other hardware is in
-[Choose local models](MODEL_SELECTION.md). For live answers about your machine, use `models {view:"catalog"}`
-or `freellama catalog`; for the exact calls, see the [cookbooks](cookbooks/README.md).
+A dated comparison of local models for each job on one machine, separating measured results
+from published claims. The method for choosing a model on other hardware is in
+[Choose local models](MODEL_SELECTION.md). For current installed candidates, use `models {view:"catalog"}`
+or `freellama catalog`; job execution first consults [machine-local calibration](MODEL_SELECTION.md#turn-measurements-into-routing-evidence),
+then catalog fallback. The tables below do not pin that live choice. For exact calls, see the [cookbooks](cookbooks/README.md).
 
 **Snapshot:** 2026-11-01. Apple M2 Pro, 32 GB unified memory, Ollama 0.40.13, FreeLlama 0.8.0. Other
 work was running throughout, leaving about 4–14 GB available. Each model answered the same questions
@@ -30,11 +31,11 @@ Group a model by the job it does, using the capability Ollama reports. FreeLlama
 | Speech | `speech` | `speech` | Text to speech | `audio {action:"speak", text}` |
 | Embedding | `embedding` | `embedding_only` | Code search, duplicates, image search | `run_task {task:"embedding", files, nearest}` |
 | Decision | `decision` | `decision` | Classify, triage, route, yes/no | `run_task {task:"decision", state, questions}` |
-| Not runnable in Ollama | none | none | Rerankers, tiny classifiers, most dedicated speech models | Needs another runtime; FreeLlama cannot serve them |
+| Not runnable in Ollama | none | none | Rerankers, tiny classifiers, most dedicated speech models | Ollama cannot run these; the [managed llama.cpp adapter](LLAMACPP.md) supports only its documented text-task subset |
 
-## Tested picks for each job
+## Historical tested picks for each job
 
-### Use now: installed, and fit alongside normal work
+### Installed at the snapshot: fit alongside normal work
 
 | Job | Model | File | Memory needed | Evidence |
 |---|---|---:|---:|---|
@@ -42,13 +43,13 @@ Group a model by the job it does, using the capability Ollama reports. FreeLlama
 | Find code for an issue | `embeddinggemma-2:740m-q8_0` | 1.3 GB | 1.4 GB | Measured: right file first 9/10, top 3 9/10; `nomic-embed-text` 6/10 and 8/10, but indexes in 42 s against 67 s |
 | Find duplicates | `nomic-embed-text:v1.5` | 0.3 GB | 1.4 GB | Measured: 8/12 real issue pairs |
 | Find an image by describing it | `embeddinggemma-2:740m-q8_0` | 1.3 GB | 1.4 GB | Measured: 5/5 screenshots, about 1 s per query |
-| OCR, reading text in images | `hf.co/openbmb/MiniCPM-OCR-1.0-gguf:Q4_K_M` | 1.4 GB | 3.8 GB | Measured: 30/30 in all 3 rounds, 257 ms median; `glm-ocr` also 30/30 at 1,885 ms |
+| OCR, reading text in images | `hf.co/openbmb/MiniCPM-OCR-1.0-gguf:Q4_K_M` | 1.4 GB | 3.8 GB | Measured: 30/30 in all 3 rounds, 257 ms median. Its tables come back as OTSL; `read-table` turns them into Markdown |
 | Image understanding | `lfm3-vl:2b-q4_K_M` | 1.5 GB | 4.1 GB | Measured: 20/20 drawn images in all 3 rounds, 299 ms; `lfm3-omni` 18/20, `bonsai-2:9b` 17/20 |
 | Speech to text | `lfm3-audio:2b-q4_K_M` | 1.7 GB | 5.5 GB | Measured: 1.7% word error on 12 generated sentences, 276 ms median, French and Spanish translation worked; `lfm3-omni` the same |
 | Text to speech | `phonon:en` | 0.1 GB | not previewed | Measured: 0.8–1.7% word error when transcribed back, 102 ms median; both LFM3 models 0% at 950 ms, but the judge is an LFM3 model |
-| Small coding tasks | `lfm3:1.7b-q4_K_M` | 1.1 GB | 3.3 GB | Measured: 21, 20, 20 of 25, 579 ms median; `bonsai-2:9b` 19, 19, 17 |
+| Small coding tasks | `bonsai-2:9b` | 3.2 GB | 10.3 GB | Measured: 19, 19, 17 of 25, 475 ms median, and every workflow case right in the 2026-11-12 field test. `lfm3:1.7b` scored 21, 20, 20 on the questions but answered workflows with tool calls nobody asked for, so it is no longer a pick |
 
-### Use when memory is free: installed
+### Installed at the snapshot: need more free memory
 
 | Job | Model | File | Memory needed | Evidence |
 |---|---|---:|---:|---|
@@ -66,7 +67,7 @@ Group a model by the job it does, using the capability Ollama reports. FreeLlama
 | `bonsai-2:27b` | 8.3 GB | Installed. FreeLlama asks for 16.3 GB available; this machine never had more than 14.4 GB, so it is not measured here. Published: SWE-bench Verified 60.8%, LiveCodeBench 90.1% |
 | `qwen3.6:27b-q4_K_M` | 17.4 GB | Needs 28.7 GB available, so only on a nearly empty 32 GB machine. Removed from this machine untested |
 
-### Worth adding: not installed, untested here
+### Candidates at the snapshot: not installed, untested here
 
 Pull a tag only with approval for that exact tag and size.
 
@@ -97,9 +98,11 @@ score) in one forward pass, with no generated text. Local decision models follow
 The original Nimble 9B (not v3) scored 75.7% against Jev 1.13's 76.0% on the same 3,880 human-labeled
 decisions; its publisher ran that test.
 
-**Recommendation:** keep `strands-decider` for everyday decisions, and trial `nimble-v3:9b-q4_K_M`
-for harder ones. Compare them on your own labeled data before switching, since `strands-decider`
-already scores 35/36 on real issues here.
+**Historical interpretation:** `strands-decider` was the smaller starting point and
+`nimble-v3:9b-q4_K_M` a candidate for harder decisions in this sample. The 35/36 issue result
+is one dataset, not a calibrated accuracy guarantee. Compare exact installed tags on your own
+labeled data and use [machine-local evidence](MODEL_SELECTION.md#turn-measurements-into-routing-evidence)
+before changing the live choice.
 
 **Limits of this comparison:**
 
@@ -111,9 +114,9 @@ Other families:
 
 - `d1`, `d1-omni` and `d3` publish other benchmarks, so they are not ranked here. `d1-omni` matters when decisions need images or audio.
 - `jevos-v4` claims 28–130 ms on a laptop CPU with its own binary and also ships GGUF files. No JevBench score was found, and loading in Ollama was not checked.
-- `openjev` is 27B and licensed CC-BY-NC, which bars commercial use.
+- `openjev` was listed as 27B under CC-BY-NC in this snapshot; inspect the current repository license before use.
 
-## New models on Hugging Face and Ollama
+## Candidates found on Hugging Face and Ollama, 2026-10-31
 
 Trending on 2026-10-31 and filtered to what could fit. Quality is the publishers' claim. **Path**:
 Ollama is an official tag, HF GGUF pulls as `hf.co/<repo>:<quant>`, and "not via Ollama" needs another
@@ -152,25 +155,28 @@ through the real CLI.
 | Model | Local questions | GitHub questions | Notes |
 |---|---:|---:|---|
 | `lfm3-audio:2b-q4_K_M` | 3, 3, 3 of 8 | 3, 3, 3 of 6 | Guessed names and repeated calls; 3.2 calls per question |
-| `lfm3:1.7b-q4_K_M` | 6, 5, 5 of 8 | 1, 1, 1 of 6 | Good on local lookups, weak on GitHub; 2 calls per question, 20 s median |
+| `lfm3:1.7b-q4_K_M` | 6, 5, 5 of 8 | 1, 1, 1 of 6 | Good on local lookups, weak on GitHub; 2 calls per question, 20 s median. No longer a pick |
 | `bonsai-2:9b` | 4, 5, 5 of 8 | 3, 5, 5 of 6 | 2.2 calls per question; 82 s median per question |
 | Nemotron-3.5-Bolt-E4B | 6, 6, 6 of 8 | 5, 5, 5 of 6 | 2.2 calls per question; 96 s median per question; needs 13.5 GB |
 
 Three rounds each. An answer counts as right only if it contains every expected fact and rests on at
-least one tool call.
+least one tool call. These historical research outcomes are separate from the newer
+[tool-calling smoke suite](RELIABILITY.md#measured-on-2026-11-14), which has two exact models and
+one trial per public case and does not establish general agent reliability.
 
 ## What runs in FreeLlama
 
 - **Managed, with routing, memory checks and receipts:** chat, coding, vision, embeddings, decisions.
 - **Through serve's passthrough, with memory checks:** speech and transcription (`audio`).
-- **Not supported:** a second, non-Ollama runtime; separate images per question in one decision, because Ollama shares images across all questions.
+- **Managed llama.cpp text tasks:** one operator-run loopback `llama-server`, selected explicitly as `name/model`; see the [supported task subset and lifecycle limits](LLAMACPP.md).
+- **Not supported:** separate images per question in one decision, because Ollama shares images across all questions.
 - **Hugging Face:** GGUF repositories work in discovery and pull through Ollama as `hf.co/<repo>:<quant>`. Multimodal GGUF needs an Ollama-supported architecture.
 
 ## Still open
 
 - Measure `bonsai-2:27b` when 16.3 GB is available.
 - Measure audio embeddings through `files`; image embeddings are measured, audio ones are not.
-- Run the candidates under "Worth adding".
+- Evaluate the dated candidates above only after checking current availability and approving any exact-tag download.
 - Test `embeddinggemma-2` on duplicates; only `nomic-embed-text` is measured there.
 
 ## Detailed results

package/docs/MODEL_FORMS_AND_RESULTS.md

@@ -1,10 +1,14 @@
 # Model forms and measured results
 
-Moved from the root README. Dated measurements describe one machine and workload.
+Dated measurements describe one machine and workload. Current job/workflow selection uses
+[machine-local calibration](MODEL_SELECTION.md#turn-measurements-into-routing-evidence) before
+catalog fallback; the historical ratings below do not determine that choice. Newer
+[tool-calling evidence](RELIABILITY.md#measured-on-2026-11-14) covers two models with one trial on
+each of seven public cases, separately from the older single-call examples below.
 
 ## Model forms
 
-Managed forms get routing, admission and a receipt; passthrough forms reach Ollama through `serve`
+Ollama-managed forms get routing, admission and a receipt; passthrough forms reach Ollama through `serve`
 with the raw stream cap and memory checks but no routing or receipt.
 
 | Form | Ollama endpoint | Through FreeLlama | Measured here |
@@ -19,21 +23,25 @@ with the raw stream cap and memory checks but no routing or receipt.
 | Transcription and translation | `/v1/audio/transcriptions`, `/v1/audio/translations` | MCP `audio`, through the passthrough's memory checks; LFM3 audio needs Ollama 0.40.12+ | `lfm3-audio:2b-q4_K_M` transcription 7.5% word error over 5 sentences; translation not measured |
 | Image generation | removed from Ollama in 0.32.6 | not available | — |
 
-Find any of these with `models {view:"library", capabilities:["decision"]}` (or `audio`,
+The [managed llama.cpp adapter](LLAMACPP.md) also supports explicit bounded text tasks; it does
+not inherit Ollama's modalities, automatic catalog discovery or placement evidence.
+
+Find Ollama forms with `models {view:"library", capabilities:["decision"]}` (or `audio`,
 `embedding`, `vision`...), or a Hugging Face GGUF build with a `query`.
 
 ### Local models, measured
 
 One Apple M2 Pro (32 GB), Ollama 0.40.8, FreeLlama 0.6.1, 2026-10-30, while other builds kept the
 one-minute load near 14 on 12 cores. Each check has a known answer; the batteries are small, so read
-the ratings as a first screen for these exact tags, not a benchmark.
+the ratings as historical editorial judgments for these exact tags, not a benchmark score or
+a measured probability of success. They predate the auditable [reliability corpus](RELIABILITY.md).
 
 | Model | Download | Forms | Result | Speed (warm) | Rating | Use it for |
 |---|---:|---|---|---|---:|---|
-| `gpt-oss:20b` | 13.8 GB | text, tools, thinking, JSON | 7/7: exact output, arithmetic, bat-and-ball, code reading, JSON extraction, tool call, 9/9 ticket routing | 1.4 s per routed ticket; 12 s first call with load | 9/10 | General agent work and tool calls when quality matters |
+| `gpt-oss:20b` | 13.8 GB | text, tools, thinking, JSON | 7/7: exact output, arithmetic, bat-and-ball, code reading, JSON extraction, tool call, 9/9 ticket routing | 1.4 s per routed ticket; 12 s first call with load | 9/10 | Candidate for the measured text tasks; broader tool reliability unmeasured in that round |
 | `nomic-embed-text:v1.5` | 274 MB | embedding | Retrieval 4/4 top-1; ticket routing 9/9 from two examples per team | 51 ms per item, batched | 9/10 | Search, deduplication, cheap classification |
 | `strands-decider:2b-nvfp4` | 1.3 GB | decision | Ticket routing 9/9, return-policy yes/no 3/3, damage rubric 2/3 | 0.17 s per question | 8/10 | Fast routing and policy checks with probabilities; rubric scores run low at the top level |
-| `glm-ocr:latest` | 2.2 GB | vision (OCR) | 4/4 phrases read from a dense report strip; 0/7 on text tasks | 11 s per strip at a 400-token cap | 8/10 OCR, 1/10 text | Reading screenshots and scans only |
+| `glm-ocr:latest` | 2.2 GB | vision (OCR) | 4/4 phrases read from a dense report strip; 0/7 on text tasks | 11 s per strip at a 400-token cap | 8/10 OCR, 1/10 text | OCR candidate for screenshots; scans were not measured |
 
 ### What they did for a coding agent
 

package/docs/MODEL_INTROSPECTION.md

@@ -1,6 +1,9 @@
 # Inspecting model precision, placement and activations
 
-Reviewed 1 November 2026 against Ollama's current API documentation and source. The locally installed CLI reports 0.40.13; this session could not reach its server, so no live model or activation was inspected. The commands below are instructions, not recorded observations.
+External API review: 1 November 2026. That session's locally installed Ollama CLI reported 0.40.13;
+its server was unreachable, so no live model or activation was inspected. The commands below are
+instructions, not recorded observations. The FreeLlama integration boundary was updated on
+14 November 2026; the historical API review is not a new verification of upstream versions.
 
 ## What “which bit/neuron was enabled” can mean
 
@@ -109,7 +112,11 @@ This example was not run. It assumes a compatible research model already loaded
 
 ## FreeLlama integration boundary
 
-Keep ordinary routing and resource control on stock Ollama. A future inspection tool can expose quantization, static tensor metadata and residency through existing endpoints. Label activation tracing unsupported unless a specific backend adapter actually provides it.
+Ordinary discovery and observed placement use stock Ollama. The optional
+[managed llama.cpp adapter](LLAMACPP.md) provides bounded text inference, not activation traces
+or verified physical placement. Neither engine path exposes hidden activations through FreeLlama.
+Static metadata and residency are separate from activation tracing; label traces unsupported
+unless a specific backend adapter actually provides them.
 
 If deeper tracing is added, use a separate explicitly enabled debug runner or an instrumented Ollama backend, with bounded capture and paged artifacts. Rust should manage jobs, limits, cancellation and trace metadata. It should not infer neuron activity from GPU utilization, memory use, token probabilities or generated reasoning text.
 

package/docs/MODEL_SELECTION.md

@@ -4,13 +4,15 @@ To offload a task, select an installed model with the required capabilities and
 then check its memory fit under the intended concurrency. FreeLlama applies model eligibility
 and admission; the calling agent still verifies the output.
 
-The measurements in this guide come from single machines and are not compiled into routing:
+The dated tables in this guide describe single machines; they do not qualify live routing on
+another host. Current job selection uses [machine-local evidence](#turn-measurements-into-routing-evidence)
+first, then the portable catalog fallback. The historical sources are:
 [Recommended starting points](#recommended-starting-points-by-form) from an Apple M2 Pro with 32 GB,
 the sections after it from an Apple M4 Pro with 48 GB unified memory. Re-run
 `npx @octocodeai/freellama doctor`, `models`, `bench-all`, and your quality suite on the target hardware.
 `doctor.machine.memory_bytes` is host RAM; a discrete GPU's usable VRAM must be observed separately.
 
-For the best model per job measured on one machine, with newer candidates, see the [local model catalog](LOCAL_MODELS.md).
+For a dated per-job comparison and candidates from one machine, see the [local model catalog](LOCAL_MODELS.md).
 
 For supported features and sourced publisher use-case guidance, inspect
 `models {view:"installed", includeLibrary:true}` through MCP. [Model metadata](#model-metadata) explains digest matching, variant capabilities,
@@ -19,7 +21,7 @@ custom-model fallbacks, and how to read family claims without treating them as q
 ```mermaid
 flowchart TD
     T{"What must the task do?"}
-    T -->|"Generate text, code, or inspect images"| L["Start with one qualified large multimodal model"]
+    T -->|"Generate text, code, or inspect images"| L["Inspect installed models with the required capabilities"]
     T -->|"Create retrieval vectors"| E["Use a small embedding model"]
     T -->|"Classify, route, or answer yes/no"| D["Use a small decision model"]
     T -->|"Transcribe or speak"| A["Use an audio or speech model"]
@@ -46,14 +48,14 @@ calls shown are the current ones. Later and larger measurements, audio and memor
 | Text, tools, JSON, reasoning | `gpt-oss:20b` (13.8 GB) | 7/7: exact output, arithmetic, bat-and-ball, code reading, JSON extraction, tool call, 9/9 ticket routing | 1.4 s per routed ticket | `run_task {model:"gpt-oss:20b", prompt, think:"low"}`; add `tools` or `format:"json"` |
 | Embeddings | `nomic-embed-text:v1.5` (274 MB) | Retrieval 4/4 top-1; ticket routing 9/9 from two examples per team | 51 ms per item, batched | `run_task {task:"embedding", files:["src/**/*.rs"] or input:[...], nearest:{query:"..."}}`: ranked labels, not vectors |
 | Decisions (classify, route, score) | `strands-decider:2b-nvfp4` (1.3 GB) | Ticket routing 9/9, return-policy yes/no 3/3, damage rubric 2/3 | 0.17 s per question | `run_task {task:"decision", model, state or files:[path], questions}`; one line per answer |
-| OCR | `glm-ocr:latest` (2.2 GB) | 4/4 phrases from a dense report strip; 0/7 on text tasks | 11 s per strip at a 400-token cap | `run_task {task:"vision", model:"glm-ocr", prompt:"Text Recognition:", files:[path]}` |
+| OCR | `hf.co/openbmb/MiniCPM-OCR-1.0-gguf:Q4_K_M` (1.4 GB) | 30/30 rendered text images in all 3 rounds | 257 ms median | `run_task {workflow:"read-text", files:[path]}` (or `job:"ocr"`) |
 
 `gpt-oss:20b` reasons before answering, and its reasoning counts as output. FreeLlama sends no
 output cap of its own and sizes the context from what the model has needed before, so leave room
 for the reasoning if you set `num_predict` yourself. It needed about 17 GB available, so FreeLlama held
-it when other work left less. `glm-ocr` reads text from images only and keeps generating after the
-visible text ends. `stopOnRepetition` handles that and is on by default without `num_predict`; set it
-explicitly if you also cap the output. The `strands-decider` tags need Apple silicon, and decision
+it when other work left less. OCR models read text from images only; asked a chat question, one keeps generating to its output
+limit. Use them through the `ocr` job's trained prompts. `stopOnRepetition` stops one that loops and is
+on by default without `num_predict`; set it explicitly if you also cap the output. The `strands-decider` tags need Apple silicon, and decision
 models answer only through Ollama's decision API, which `run_task {task:"decision"}` reaches with
 FreeLlama's routing and memory checks. A busy CPU does not refuse them; low memory does.
 
@@ -124,7 +126,7 @@ These examples do not establish that a tag is installed or suitable on your host
 
 Dedicated OCR models answer best to the prompts their model card documents; name the image in
 `run_task {files}` rather than pasting base64 `images`. A model that repeats its answer until its
-output limit stops early with `stopOnRepetition`, on by default without `num_predict`. See [Route images](https://github.com/bgauryy/FreeLlama/blob/main/packages/mcp/README.md#route-images)
+output limit stops early with `stopOnRepetition`, on by default without `num_predict`. See [Loop control](FEATURES.md#loop-control)
 for the early stop and the `output_check` receipt.
 
 `muse-glimmer:30b-mlx` was a credible alternative and answered 29 of 30 questions correctly
@@ -166,6 +168,33 @@ this machine and workload, not a universal model-placement rule. Follow
 
 ## Turn measurements into routing evidence
 
+Run `freellama bench-all --include EXACT_INSTALLED_TAG` on the serving machine, repeating
+`--include` for other installed tags. The default output is `benchmark-report.json` in the
+FreeLlama home (`FREELLAMA_HOME`, otherwise `~/.octocode/freellama`), which `serve` discovers on
+its next start. `--output` chooses a different report path. No models are downloaded. Existing
+resident models are left loaded; only models this run successfully warmed are eligible for cleanup.
+
+Schema 2 reports record an observation timestamp, server version, artifact digests, OS, CPU,
+RAM, available GPU identity and a domain-separated hash of the OS machine identifier. Raw machine
+identifiers and hostnames are not included. Routing requires the same host/hardware signature,
+exact endpoint (apart from a trailing slash), and current model digest. Remote endpoints, unknown
+machine identity, reports from another host, and older reports remain readable but do not qualify
+local throughput. Regenerate them on this machine. Primary-endpoint scores never qualify a separate
+CPU-assigned runner. GPU details absent from the OS remain unknown; server settings and upgrades can
+change performance without changing this signature, so rerun after those changes.
+
+Job and workflow selection first asks the Rust router for a locally benchmarked model. Ordinary
+jobs request the fastest route; capable workflows require a task quality policy as well. OCR also
+requires quality evidence, and image search requires vision-capable embeddings. Without matching
+evidence, selection falls back to installed catalog picks and their live memory preview. If no
+tested catalog pick is installed, a capability-qualified installed model may run with an untested
+warning. A conservative preview miss can still reach the authoritative runtime admission gate,
+which admits, waits or refuses; selection never reserves memory. An explicit caller `model` wins.
+The result states when local throughput guided the choice; a throughput screen alone does not
+establish job accuracy. `models {view:"catalog"}` shows portable starting points, not the final
+calibrated decision. This calibrated-first path applies to `run_task` jobs/workflows and virtual
+gateway models; `delegate_research` has its own [catalog-and-routing sequence](cookbooks/research-agent.md).
+
 `bench-all` measures local performance. A quality benchmark measures correctness. FreeLlama needs
 both the generated task policy and local benchmark data before grading a route medium confidence.
 See the [CLI reference](CLI.md#earn-medium-routing-confidence) for the commands.
@@ -211,7 +240,7 @@ feature with `capability`, `use`, `reason`, and `basis:"supported_feature"`;
 | Feature | Supported workflow | What to evaluate separately |
 |---|---|---|
 | `completion` | Text generation | Coding, summarization, reasoning, and instruction-following accuracy |
-| `tools` | Tool calls | Argument correctness, tool selection, and agent-loop reliability |
+| `tools` | Tool calls | Argument correctness, tool selection, and agent-loop reliability; see the [bounded public smoke evidence](RELIABILITY.md#public-tool-calling-smoke-suite) |
 | `vision` | Image input | OCR and visual understanding accuracy |
 | `audio` | Audio workflows advertised by the model | Supported audio task and accuracy |
 | `embedding` | Retrieval and similarity vectors | Retrieval quality on your documents |

package/docs/MONITORING.md

@@ -16,13 +16,15 @@ token and keeps it only for the browser tab.
 
 | Endpoint | CLI | MCP | What it returns |
 |---|---|---|---|
+| `GET /health` | | `doctor` (`platform_health`) | Liveness, admission capacity, and configured backends; answers as soon as serve listens |
+| `GET /ready` | `freellama ready` | `doctor` (`readiness`) | 200 once the primary Ollama (and CPU backend, when configured) answers and its catalog reads; otherwise 503 with `waiting_for` |
 | `GET /status` | `freellama status` | `doctor {view:"status"}` | Live view of every backend, the raw proxy, loaded models, host, Ollama settings, and today's usage |
 | `GET /usage?days=N` | `freellama usage --days N` | `doctor {view:"usage", days:N}` | Totals per day and per model (default 7 days) |
 | `GET /metrics` | | | Prometheus text exposition |
 | `GET /config` | `freellama config` | | Effective runtime settings and the source of each |
 | `POST /config/reload` | `freellama config --reload` | | Re-read the runtime file now; `422` keeps the last good values |
 | `GET /jobs` | `freellama jobs` | `task_jobs {action:"list"}` | Deferred job metadata without prompts or retained results |
-| `GET /jobs/:id` | `freellama jobs --id <uuid>` | `task_jobs {action:"get", jobId}` | One job receipt and its retained result or error |
+| `GET /jobs/:id` | `freellama jobs --id <uuid>` | `task_jobs {action:"get", jobId}` | One job receipt and its retained result or error. `?wait_ms=N` (at most 5000) holds the read until the job changes; the reply then carries `long_poll: true` |
 | `POST /jobs/:id/cancel` | `freellama jobs --id <uuid> --cancel` | `task_jobs {action:"cancel", jobId}` | Cancellation receipt after local permits are released |
 | `DELETE /jobs/:id` | `freellama jobs --id <uuid> --remove` | `task_jobs {action:"remove", jobId}` | Stops active work, releases local permits, and removes the retained record |
 
@@ -49,9 +51,20 @@ token and keeps it only for the browser tab.
   A remote endpoint is never probed and is reported as unknown.
 - **`usage_today`**: tasks, errors, tokens, and busy time since midnight UTC.
 - **`task_jobs`**: deferred task IDs, selected models, configured backends, priorities, total
-  deadlines, states, and waiting reasons. States cover discovery, admission, resources, runner
+  deadlines, states, and waiting reasons. MCP `doctor {view:"status"}` keeps only the active jobs
+  plus counts by state; `task_jobs {action:"list"}` lists them all. States cover discovery, admission, resources, runner
   transitions, loading, execution, and completion. Metadata excludes the original payload.
 
+On Windows, host memory comes from a bounded, unprivileged PowerShell CIM query. The
+`windows_cim` observation converts `Win32_OperatingSystem.TotalVisibleMemorySize` and
+`FreePhysicalMemory` from KiB to bytes; available memory is the latter, with kind
+`windows_free_physical_memory`. Missing, malformed, overflowing, or inconsistent readings remain
+unknown. See [Microsoft's field definitions](https://learn.microsoft.com/en-us/windows/win32/cimwin32prov/win32-operatingsystem).
+The default `require-memory` policy holds admission when this observation fails, as on Linux and
+macOS. CPU load, OS pressure, thermal, swap and GPU measurements remain unavailable on Windows;
+`require-all` therefore holds admission. An operator can explicitly choose `best-effort` to admit
+with unknown memory telemetry; observed low memory still follows the configured low-memory policy.
+
 A task receipt explains its own wait. `admission.mode` is `resident_shared` (the runner was
 reused without the backend load lock) or `nonresident_transition_exclusive` (this task loaded the
 runner under it). `admission.reason` says why a present runner could not be shared when its identity
@@ -66,7 +79,9 @@ Use `run_task {defer:true}` in MCP or `task --defer` in the CLI to get a job han
 status request omits results; fetching one ID returns its retained result or structured failure.
 Terminal states are `completed`, `failed`, `cancelled`, and `expired`. Cancellation is per task,
 waits for local permits to be released, and does not unload a shared runner. Process-local job
-retention limits are documented in [CLI controls](CLI.md#execute-tasks).
+retention limits are documented in [CLI controls](CLI.md#execute-tasks). The optional durable-job
+mode retains receipts/results in the file-backed store without replaying inference; its recovery
+and idempotency rules are in [durable deferred jobs](DEFERRED_JOBS.md).
 
 Removal uses the same exact job ID. It cancels active work before discarding the record, returns
 `{id, removed:true, status, scope:"process_memory"}`, and omits the saved result. The ID then returns
@@ -87,6 +102,13 @@ including after an Ollama restart. Each config receipt includes `observed_at` an
 invalidates the use of measurements taken under different parallelism or KV cache settings.
 FreeLlama admission limits remain governed by its runtime configuration.
 
+These Ollama residency and process-setting observations do not apply to the optional managed
+llama.cpp adapter. Its receipt identifies `execution.engine:"llamacpp"`, an operator-configured
+memory forecast and processor assignment, with physical placement unknown. Its bounded text
+tasks share admission pools and the host-memory governor; their results do not train Ollama
+placement, residency or preload facts. See the [engine contract](LLAMACPP.md) for supported tasks
+and the single live protocol smoke's limits.
+
 `host.gpu_activity` is nullable. When available, it identifies its `source`,
 `scope: "observed_devices_all_processes"`, `window: "driver_defined"`, and individual `devices`.
 Each device has an identity, nullable `busy_percent`, and nullable `memory_busy_percent`. Apple
@@ -96,14 +118,17 @@ Use `host.sample_age_ms` to check freshness.
 
 NVIDIA activity comes from the same bounded `nvidia-smi` query as memory. Apple Silicon activity
 comes from the accelerator's `PerformanceStatistics` in a bounded IORegistry read. Those Apple
-properties are driver-specific and may disappear or change; the read can cover only the devices
-within its retained output. AMD, Windows, unsupported drivers, and unavailable reads report unknown activity. The counters include other processes, use a driver-defined observation
+properties are driver-specific and may disappear or change. A sample that may have reached the
+64 KiB output cap or omits a recognized device's busy counter is discarded rather than treated as
+complete. Malformed NVIDIA device rows likewise discard the activity sample. AMD, Windows, unsupported drivers, and unavailable reads report unknown activity. The counters include other processes, use a driver-defined observation
 window, and do not attribute activity to a FreeLlama task or establish bandwidth or energy
-efficiency. They are diagnostics and do not alter admission or pressure decisions.
+efficiency. They are diagnostic by default. The opt-in GPU headroom guard below also uses valid
+busy counters to hold new GPU work.
 
 Each backend's admission `queue_depth` includes both `slot_waiters` and `resource_waiters`.
-A resource waiter returns its execution slot after two seconds but remains in the bounded queue
-until capacity recovers, its deadline expires, or it is cancelled.
+An Ollama resource waiter returns its execution slot after at most two seconds; llama.cpp yields
+after its initial bounded probe. Both remain in the bounded queue until capacity recovers, their
+deadline expires, or they are cancelled.
 
 Model affinity is bounded: `affinity_max_skips` (4) and `affinity_max_wait_ms` (2000) in the same
 receipt are how often, and how long, an older waiter can be passed over before its class is served
@@ -116,6 +141,20 @@ process-wide memory for request bodies being read or held by running requests,
 queue wait and is then refused with 503 `request_body_budget_exhausted`; raise it, or lower
 `FREELLAMA_MAX_REQUEST_BYTES`, when large uploads are normal.
 
+Managed response buffering also uses `FREELLAMA_MAX_REQUEST_BYTES` (64 MiB by default) as a
+per-response byte ceiling. Ollama JSON, embedding and decision responses, error bodies, and total
+NDJSON stream bytes share this limit. Oversized Ollama output returns `502` with
+`code:"upstream_response_too_large"`, `retry_safe:false` and `execution_outcome:"unknown"`;
+do not replay it automatically. This ceiling is separate from the shared request-body budget
+and does not represent total process heap usage. See [stream bounds](ARCHITECTURE.md#request-shapes)
+for backpressure and cancellation behavior.
+
+The management catalog and compatibility proxy share a separate 4 MiB cap for internal model lists,
+`/api/show`, residency observations and server versions. Oversized or failed observations remain
+unknown; they cannot establish a model's residency or allocation forecast. An oversized external
+engine model list appears under `unreachable` in the merged list. Direct raw metadata requests
+still stream their upstream responses unchanged.
+
 ## Record usage
 
 Every managed task, and every raw Ollama or OpenAI-compatible generation, updates in-memory usage
@@ -167,7 +206,7 @@ FreeLlama waits for safe capacity within a deadline and bounds the retained back
 | Managed queue full | `429` | `admission_queue_full` |
 | Raw proxy over its cap after `raw_queue_wait_seconds` | `429` | |
 | Managed wait exceeded `max_queue_wait_seconds` (or the task's `max_wait_seconds`) | `503` | `admission_timeout` |
-| OS pressure, swapping, CPU, thermal, or required telemetry still holds when the waiting budget expires (low available RAM alone only warns on macOS; low RAM and swapping never hold an already-resident runner, and CPU load and thermal never hold one on the GPU) | `503` | `resource_admission_unavailable` |
+| A new load still does not fit available memory with its reserve, or OS pressure, swapping, CPU, thermal, or required telemetry still holds, when the waiting budget expires. Low available RAM refuses by default on every OS; `--low-memory advisory` (`FREELLAMA_LOW_MEMORY=advisory`) makes it a warning. Low RAM and swapping never hold an already-resident runner, and CPU load and thermal never hold one on the GPU | `503` | `resource_admission_unavailable` |
 | Backend circuit open | `503` | `upstream_circuit_open` |
 
 These capacity refusals carry a `Retry-After` header and `retry_after_seconds` in the body. For a queue it
@@ -176,6 +215,63 @@ task's cost), and the backend's recent average service time, which excludes queu
 seconds). A task can ask for a shorter wait with `max_wait_seconds` (`maxWaitSeconds`
 in MCP); the server caps it at the configured maximum.
 
+## Leave capacity for other applications
+
+Resource thresholds belong to the operator, not to a model name or a fixed machine size. Start
+`serve` with CPU and, optionally, GPU headroom targets:
+
+```bash
+freellama serve \
+  --resource-hold-load-per-cpu 0.75 \
+  --resource-resume-load-per-cpu 0.5 \
+  --resource-hold-gpu-busy-percent 80 \
+  --resource-resume-gpu-busy-percent 60
+```
+
+These are **admission targets, not utilization quotas**. They hold new work until observations
+recover; they do not pause, kill, or throttle work already executing. One admitted inference can
+still saturate a device. Use OS/container CPU quotas and upstream/device isolation when a hard
+resource partition is required. Direct calls to Ollama bypass FreeLlama's guard.
+
+| Setting | Default | Meaning |
+|---|---|---|
+| `--resource-hold-load-per-cpu` | `1.5` | Hold CPU-bound work when one-minute load / logical CPUs reaches this value |
+| `--resource-resume-load-per-cpu` | `1.0` | Recover the CPU hold at or below this value |
+| `--resource-hold-gpu-busy-percent` | Disabled | Hold GPU work when any observed device reaches this busy percentage |
+| `--resource-resume-gpu-busy-percent` | Disabled | Recover only when all observed devices are at or below this percentage |
+
+The four flags also accept environment variables named `FREELLAMA_RESOURCE_HOLD_LOAD_PER_CPU`,
+`FREELLAMA_RESOURCE_RESUME_LOAD_PER_CPU`, `FREELLAMA_RESOURCE_HOLD_GPU_BUSY_PERCENT`, and
+`FREELLAMA_RESOURCE_RESUME_GPU_BUSY_PERCENT`. They are startup settings, not runtime-TOML keys;
+changing them requires restarting your own FreeLlama service. Inspect the effective policy under
+`host.policy` in raw `status` (or the full saved `doctor` response).
+
+CPU load is **not CPU utilization**: `0.75` does not guarantee 25% idle CPU. It scales with logical
+CPU count and includes the OS's runnable/waiting load. Both CPU thresholds must be finite and
+positive, with resume less than hold; no fixed upper ceiling is imposed. Missing CPU telemetry
+follows `--resource-telemetry-policy`; use `require-all` if unavailable CPU-load observations must
+block work. CPU load holds CPU-assigned work and work on hosts without an observed accelerator;
+it does not hold an observed GPU runner merely because a compiler is busy. Raw Ollama requests
+with explicit integer `options.num_gpu: 0` use CPU admission without rewriting the body; this is
+request intent, not proof of physical placement. Unknown raw/OpenAI-compatible placement stays
+conservatively GPU-gated when the optional GPU guard is enabled.
+
+Both GPU settings must be supplied together and satisfy `0 <= resume < hold <= 100`. GPU activity
+is optional by default so unsupported machines continue to work. When explicitly enabled, missing,
+partial, or invalid GPU busy counters refuse GPU admission with `gpu_activity` in the assessment's
+`missing_telemetry`, even with `best-effort`; CPU-assigned work remains eligible. Enable this guard
+only on hosts exposing the counters described above. It conservatively considers the busiest of
+all observed GPUs, not a per-model device assignment. CPU-only or remote Ollama deployments should
+enforce resource controls on the machine actually executing work: remote upstreams bypass local
+resource admission. Holds recover after two fresh healthy samples by default, avoiding rapid
+stop/start at a single threshold.
+
+For memory headroom, the existing `--resource-hold-available-percent` and
+`--resource-resume-available-percent` settings default to 15 and 20. The per-load reserve is
+size-aware rather than a promise to keep that entire share unused for every small model. Use
+`model_memory_budget` in the runtime file to cap combined model residency on unified-memory/CPU
+backends. GPU busy percentage is not a VRAM reserve.
+
 ## Circuit breaker
 
 After `breaker_failures` consecutive upstream failures on a backend (connection errors, `500`,
@@ -187,9 +283,9 @@ model do not count. Set `breaker_failures = 0` to disable it.
 ## Adaptive concurrency
 
 Managed task costs are configurable through the runtime file's `[task_costs]` table. Valid keys
-are `completion`, `coding`, `code_repair`, `tools`, `browser`, `vision`, `embedding`, and
+are `completion`, `coding`, `code_repair`, `tools`, `browser`, `vision`, `embedding`, `decision`, and
 `long_context`. Each value is a positive integer through 4294967295. Omitted kinds retain their
-defaults: vision four, embeddings one, and other tasks two. Embeddings multiply their base by
+defaults: vision four, embeddings and decisions one, and other tasks two. Embeddings multiply their base by
 `ceil(input_items/4)`; the acquired charge cannot exceed the pool's current limit.
 
 For a qualified light OCR workload, `[task_costs]` with `vision = 2` permits two vision requests
@@ -245,7 +341,9 @@ Omitted managed `keep_alive` values use finite retention informed by measured re
 starting from `base_seconds` (300, Ollama's own default) and shortened by `pressure_factor` under a
 host hold. Explicit values take precedence. The background pressure sweep releases, under
 FreeLlama's own low-memory estimate, only runners idle longer than `base_seconds ×
-pressure_factor` (75 s by default); swapping or OS memory pressure releases any idle runner. For governed prewarming, profile compatibility, and retention
+pressure_factor` (75 s by default); swapping or OS memory pressure shortens that to `min_seconds`
+(15 s). A runner counts as used when a task ran on it here or when another client moved its Ollama
+`expires_at` since the sweep's last check; the audit then lists it as `used_elsewhere`. For governed prewarming, profile compatibility, and retention
 settings, see [Scope history and model warming](SCOPES_AND_WARMING.md).
 
 When a model about to load needs room, FreeLlama unloads idle runners itself (`keep_alive: 0`)