@octocodeai/freellama-mcp-server 0.6.2 → 0.9.0

package/docs/MONITORING.md

@@ -31,8 +31,10 @@ token and keeps it only for the browser tab.
 `status` answers "why is my task waiting?" in one call:
 
 - **`backends.gpu` / `backends.cpu`**: admission (units in use, current limit, ceiling, queue depth,
-  oldest wait, refusal and timeout counters), the adaptive limiter state, and the circuit breaker
-  (`closed`, `open` with seconds remaining, or `half_open`).
+  oldest wait, refusal and timeout counters, and `affinity_promotions`: grants given ahead of older
+  same-priority work because the waiter's model was already running or admitted last), the
+  adaptive limiter state, and the circuit breaker (`closed`, `open` with seconds remaining, or
+  `half_open`).
 - **`raw_proxy`**: active and waiting raw streams against their cap and wait budget.
 - **`loaded_models`**: Ollama's `/api/ps` with total and VRAM bytes, context length, expiry, and
   whether the model is pinned.
@@ -50,6 +52,16 @@ token and keeps it only for the browser tab.
   deadlines, states, and waiting reasons. States cover discovery, admission, resources, runner
   transitions, loading, execution, and completion. Metadata excludes the original payload.
 
+A task receipt explains its own wait. `admission.mode` is `resident_shared` (the runner was
+reused without the backend load lock) or `nonresident_transition_exclusive` (this task loaded the
+runner under it). `admission.reason` says why a present runner could not be shared when its identity
+was not established, for example `resident_identity_unobserved` when `/api/ps` reports no digest;
+it is `null` for a plain match. `queue_wait_ms`, `transition_wait_ms` (waiting for the load lock) and
+`resource_wait_ms` add up to `total_wait_ms`. Waiting for the lock is bounded by the task's queue
+deadline; a raw load that waits too long gets 503 `proxy busy`, and a raw generation whose
+allocation cannot be forecast gets 503 `model_forecast_unavailable`. See
+[Load lock and use lease](CPU_GPU_ROUTING.md#load-lock-and-use-lease).
+
 Use `run_task {defer:true}` in MCP or `task --defer` in the CLI to get a job handle. A list or
 status request omits results; fetching one ID returns its retained result or structured failure.
 Terminal states are `completed`, `failed`, `cancelled`, and `expired`. Cancellation is per task,
@@ -64,7 +76,10 @@ cancellation closes the upstream request; physical computation may continue on a
 `ollama.config` describes the primary process; `ollama.cpu_config` describes the CPU process when
 configured. Process attribution matches `OLLAMA_HOST` to the selected endpoint, so separate
 parallelism and KV cache settings size each backend independently. Fallback sources remain visible
-when process settings cannot be observed.
+when process settings cannot be observed. Each config's `effective` block reports the resolved
+`num_parallel`, `context_length`, `flash_attention` (`on`, `off`, or `auto` when unset) and the KV
+cache it sizes for (`kv_cache_type`, `kv_cache_percent_of_f16`); see
+[KV cache and flash attention](TUNING.md#kv-cache-and-flash-attention).
 
 Process settings are cached for five seconds and refreshed on discovery or monitoring requests,
 including after an Ollama restart. Each config receipt includes `observed_at` and
@@ -90,12 +105,28 @@ Each backend's admission `queue_depth` includes both `slot_waiters` and `resourc
 A resource waiter returns its execution slot after two seconds but remains in the bounded queue
 until capacity recovers, its deadline expires, or it is cancelled.
 
+Model affinity is bounded: `affinity_max_skips` (4) and `affinity_max_wait_ms` (2000) in the same
+receipt are how often, and how long, an older waiter can be passed over before its class is served
+in arrival order again. A rising `affinity_promotions` with flat queue waits means fewer model
+switches; see [the admission design](ARCHITECTURE.md#managed-task-execution).
+
+`GET /health` reports `admission.request_body_budget` (`total_bytes`, `available_bytes`): the
+process-wide memory for request bodies being read or held by running requests,
+`FREELLAMA_MAX_BUFFERED_BYTES` (256 MiB by default). When it is full, a new upload waits up to the
+queue wait and is then refused with 503 `request_body_budget_exhausted`; raise it, or lower
+`FREELLAMA_MAX_REQUEST_BYTES`, when large uploads are normal.
+
 ## Record usage
 
-Every managed task updates in-memory usage totals and offers one record to a bounded writer queue.
-The record contains time, model, backend, task kind, priority, outcome, HTTP status, prompt and
-output tokens, duration, queue wait, load time, and output tokens per second. It contains no prompt
-or response text. One worker writes records in order; task completion does not wait for disk.
+Every managed task, and every raw Ollama or OpenAI-compatible generation, updates in-memory usage
+totals and offers one record to a bounded writer queue. The record contains time, model, backend,
+task kind (`raw` for passthrough, whose `priority` field holds the path), priority, outcome, HTTP
+status, prompt, cached prompt and output tokens, duration, queue wait, load time, output tokens per
+second and, with client keys, the client. It contains no prompt or response text. Raw counts are read
+from the fields each response reports (`prompt_eval_count`, `prompt_eval_cached_count`, `eval_count`,
+OpenAI `usage`, decision `usage`) as it streams through unchanged; a client that leaves early records
+what arrived. `cached_prompt_tokens` totals show how much prompt work the runners' prompt caches
+saved. One worker writes records in order; task completion does not wait for disk.
 
 The writer queue defaults to 128 records. Configure its positive startup-only capacity through
 `usage_queue_capacity` or `FREELLAMA_USAGE_QUEUE_CAPACITY`; runtime reload does not resize the worker.
@@ -105,13 +136,16 @@ failure records a failed write. Inspect `usage_ledger` in `status`, or `ledger`
 `asynchronous_best_effort`: pending records can be lost on a crash, and in-memory totals can exceed
 persisted totals. Error evidence remains visible after later successful writes.
 
-`serve` replays a bounded regular-file ledger at startup and rotates it to `usage.jsonl.1` at
-16 MiB. Successful writes contribute to restart totals; completion is not a zero-loss persistence
-promise. Embedded callers can use the bounded `flush_ledger` barrier for orderly teardown; it
-confirms attempted writes, not recovery of dropped/failed records or a filesystem sync.
-
-The ledger lives next to the feedback file in the platform data directory. Choose another path with
-`--usage-file` (or `FREELLAMA_USAGE_FILE`), or keep totals in memory only with `--ephemeral-usage`.
+The ledger is the `usage` table of the shared database (`~/.octocode/freellama/freellama.sqlite`). `serve`
+keeps daily totals for the last 35 days in memory, computed in the database at startup, and never
+deletes records. `usage --days N` past 35 days reads the older days from the database on request
+(`retention.older_days: "ledger"`); with `--ephemeral` they are `"not_retained"`. Memory holds at
+most 256 models per day, and the task counters at most 256 models: raw requests can name any
+model, so further names are counted under `other`. Successful writes
+contribute to restart totals; completion is not a zero-loss persistence promise. Embedded callers
+can use the bounded `flush_ledger` barrier for orderly teardown; it confirms attempted writes, not
+recovery of dropped/failed records. `--home` moves the database; `--ephemeral` keeps totals in
+memory only.
 
 ## Scrape metrics
 
@@ -120,7 +154,7 @@ The ledger lives next to the feedback file in the platform data directory. Choos
 `freellama_load_seconds_total`, `freellama_raw_requests_total`, `freellama_evictions_total`,
 `freellama_circuit_open_total`, `freellama_concurrency_limit_changes_total`); task counters are
 labelled by model, backend, task, and outcome, raw requests by kind and outcome, and the rest by
-backend. Gauges cover admission, queues, the raw proxy, host and GPU memory,
+backend. `freellama_admission_affinity_promotions_total` counts model-affinity grants per backend. Gauges cover admission, queues, the raw proxy, host and GPU memory,
 loaded models, and `freellama_ollama_num_parallel`. Point Prometheus, Grafana Agent, or any
 OpenMetrics-compatible scraper at it.
 
@@ -167,8 +201,8 @@ A task captures its cost when execution starts. Reload affects tasks that start
 already queued or active tasks keep their captured cost. `config` reports effective values and
 the runtime-file source; invalid keys, zero,
 or overflow reject the reload and retain the last good configuration. Preview advice, health
-costs, and execution receipts use the resolved policy. Health retains its summary aliases and
-exposes the complete task-kind map under `admission.costs.base_by_task`. Task kinds remain
+costs, and execution receipts use the resolved policy. Health exposes the complete task-kind map
+under `admission.costs.base_by_task` and the embedding batch formula. Task kinds remain
 caller-declared; these weights are cooperative scheduling policy, not workload classification.
 See the
 [runtime example](https://github.com/bgauryy/FreeLlama/blob/main/freellama.runtime.example.toml) for the table syntax.
@@ -198,10 +232,8 @@ report verified placement and `feedback.accepted:true` while
 `execution.throughput_learning.eligible:false`; it contributes no comparable speed sample.
 
 Each backend retains at most one adaptive profile window per task kind. Status exposes
-`adaptive.profile_windows` keyed by task; `baseline_output_tokens_per_second` is a compatibility
-output-only summary by model and can omit additional task windows for the same model. Embedding
-windows report their explicit input-rate kind separately. Use the task windows for learning
-evidence. See [automatic feedback](CPU_GPU_ROUTING.md#understand-automatic-feedback) for
+`adaptive.profile_windows` keyed by task. Embedding windows report their explicit input-rate kind
+separately. Use the task windows for learning evidence. See [automatic feedback](CPU_GPU_ROUTING.md#understand-automatic-feedback) for
 persisted routing windows.
 
 The CPU backend is adaptive by default because oversubscribing it shows up as a throughput collapse
@@ -209,14 +241,24 @@ rather than an error. Every change is counted in metrics and shown in `status`.
 
 ## Model residency and eviction
 
-Omitted managed `keep_alive` values use finite retention informed by measured reuse and loading.
-Explicit values take precedence. For governed prewarming, profile compatibility, and retention
+Omitted managed `keep_alive` values use finite retention informed by measured reuse and loading,
+starting from `base_seconds` (300, Ollama's own default) and shortened by `pressure_factor` under a
+host hold. Explicit values take precedence. The background pressure sweep releases, under
+FreeLlama's own low-memory estimate, only runners idle longer than `base_seconds ×
+pressure_factor` (75 s by default); swapping or OS memory pressure releases any idle runner. For governed prewarming, profile compatibility, and retention
 settings, see [Scope history and model warming](SCOPES_AND_WARMING.md).
 
 When a model about to load needs room, FreeLlama unloads idle runners itself (`keep_alive: 0`)
-instead of letting the load wait, fail, or spill to CPU. It does this in three cases: host RAM
-can fall below the reserve, the host is already holding for low memory, or (discrete GPU) the
-model does not fit in free VRAM, where Ollama otherwise chooses its own victim.
+instead of letting the load wait, fail, or spill to CPU. It does this in four cases, checked in
+this order: (discrete GPU) the model does not fit in free VRAM, where Ollama otherwise chooses its
+own victim; the load would take the backend's loaded models past `model_memory_budget`; host RAM
+can fall below the reserve; or the host is already holding for low memory. Raw passthrough loads
+(an Ollama API call for a model that is not loaded) take the same steps, and raw calls in flight
+count as busy, so their models are not unloaded under them.
+
+`model_memory_budget` (runtime file, or `FREELLAMA_MODEL_MEMORY_BUDGET`) is `"75%"` of RAM by
+default; set a size such as `"24GiB"`, another share, or `"off"`. It applies where the runners share
+host memory: unified memory (Apple silicon) or a CPU backend.
 
 Which runners go is a small optimisation, in the spirit of llama-swap's `evict_costs` and LocalAI's
 busy-aware LRU:
@@ -226,7 +268,7 @@ busy-aware LRU:
 - **Not for a model bigger than the GPU:** it spills to the CPU whatever is unloaded, so VRAM is
   left alone.
 - **Cost of unloading a runner** = its reload time (Ollama's measured `load_duration` from earlier
-  loads, or size ÷ 1.5 GB/s before the first one) × (1 + recent uses, decaying with a 30-minute
+  loads, kept in the shared database across restarts, or size ÷ 1.5 GB/s before the first one) × (1 + recent uses, decaying with a 30-minute
   half-life; only tasks served count) × its `eviction_costs` weight (default 1).
 - **Choice:** the set of runners with the lowest total cost that frees enough memory; ties go to
   fewer runners, then less memory freed beyond what is needed. Only the memory that matters is
@@ -234,10 +276,43 @@ busy-aware LRU:
   memory.
 
 Every managed receipt that caused an eviction carries the plan under
-`memory_reservation.eviction`: what was unloaded, its cost and reload estimate, and what was kept
-and why. `status` and the page show the latest one that unloaded something. Eviction runs once per
-task, after admission and before the memory wait, and is never cancelled halfway by a deadline.
-Set `evict_idle_models = false` to leave residency entirely to Ollama.
+`memory_reservation.eviction`: what was unloaded, its cost and reload estimate, what was kept and
+why, the `trigger` (`gpu_memory`, `model_memory_budget` or `host_memory`) and `unload_ms`. `status`
+and the page show the latest one that unloaded something. Eviction runs once per task, after
+admission and before the memory wait, and is never cancelled halfway by a deadline. Set
+`evict_idle_models = false` to leave residency entirely to Ollama.
+
+`memory_reservation` names the `kv_cache_type` and `flash_attention` state it sized for. With a
+quantized cache it also carries `ollama_f16_preflight_bytes`, the f16-cache size Ollama's own fit
+check predicts. On a CPU or
+unified-memory backend, `ollama_preflight_threshold_bytes` is 80% of the available memory the load
+was admitted against; when the reserved estimate fits under it and the f16 figure does not, the
+receipt adds `ollama_may_evict_or_offload: true` and a one-line `ollama_preflight_reason`: Ollama may
+unload other runners or offload layers even though FreeLlama found room. Run previews carry the
+same fields. The reservation itself is unchanged: measurements still take precedence over the
+estimate.
+
+### The residency audit
+
+Every load, unload and eviction is recorded with its cause and duration in the shared database, so
+the audit covers every FreeLlama process on the machine. `status` summarizes the last hour under
+`residency`:
+
+| Field | Meaning |
+|---|---|
+| `model_memory_budget` | The setting, its bytes, and per backend whether it applies, what its loaded models hold, and the share used |
+| `audit.counts`, `audit.by_trigger` | Loads, unloads and evictions; causes such as `managed_task`, `raw_request`, `caller_unload`, `model_memory_budget`, `host_memory` |
+| `audit.switch_ms` | Per model: measured loads, average, total and last load time (Ollama's `load_duration`) |
+| `audit.thrashing` | Models loaded 3 or more times in 10 minutes: raise the budget, pin the model, or give it its own backend |
+| `audit.recent` | The 20 newest events (the database keeps every event; counts cover the whole hour) |
+
+A raw passthrough load is recorded when its response ends, with the `load_duration` Ollama
+reported in it, and that time also teaches the eviction planner what reloading the model costs.
+Preloads appear as `preload` or `preload_failed` events with trigger `startup` or
+`predicted_after:<model>`.
+
+A task refused because the host stayed under pressure says which signals held it:
+`resource_admission.held_by` (such as `["thermal_throttling"]`), also named in the error text.
 
 ## Who decides what: FreeLlama and Ollama
 
@@ -253,8 +328,8 @@ Two schedulers that both guess at memory can disagree. The split is:
 | Context size | Ollama by default | `num_ctx` is left to Ollama when its default covers the request |
 
 To keep the estimates honest, FreeLlama learns from every runner Ollama reports as resident,
-including ones loaded by raw clients, and saves those measurements to `footprints.json` next to the
-usage ledger. After a restart a model it has seen before is sized from Ollama's measurement, not a
+including ones loaded by raw clients, and saves those measurements in the shared database, keeping
+the larger size when two processes measure the same runner. After a restart a model it has seen before is sized from Ollama's measurement, not a
 formula. Each measurement records the `OLLAMA_NUM_PARALLEL` and KV-cache type it was taken under and
 is used only while those match, so changing either setting falls back to the formula until Ollama
 measures again. A larger context than any measured one is sized as the measurement plus exactly the extra
@@ -275,6 +350,13 @@ on a discrete GPU only) Ollama's VRAM tier, from total VRAM minus `OLLAMA_GPU_OV
 Memory estimates always use the context Ollama can allocate. Use `"explicit"` to always
 send `num_ctx`.
 
+The context-shift setting is part of a runner's identity too: Ollama reloads a runner when a
+request's `shift` differs from the one it was loaded with, and its default is on for most models.
+`context_shift = "off"` (the default) sends `shift: false` on managed chat, so a generation that
+reaches the context limit stops instead of discarding its earliest tokens, at the cost of a reload
+when raw clients use the same model. `"ollama_default"` omits `shift` so raw and managed chat share
+the runner. Both modes send `truncate: false`. `status` reports it under `ollama.context_shift`.
+
 ## Change settings without a restart
 
 Put any of the settings above in a TOML file and start `serve --runtime-config <file>` (or set

package/docs/PRODUCTION.md

@@ -11,8 +11,8 @@ not provide tenant isolation or billing.
 ## Meet the prerequisites
 
 - Install one Ollama CLI and server version. Run `freellama doctor` and resolve any mismatch.
-- Install Node.js 20 or later for MCP and npm distribution use.
-- Use the release artifacts for your operating system, or build with Rust 1.88 or later.
+- Install Node.js 22 or later for MCP and npm distribution use.
+- Use the release artifacts for your operating system, or build with Rust 1.89 or later.
 - Preinstall and benchmark exact model tags. Discovery never authorizes a pull.
 - Keep the primary and optional CPU Ollama processes on distinct loopback sockets.
 - In a slim or distroless container, install the CA bundle (`ca-certificates`) or set
@@ -38,10 +38,19 @@ The repository has no CI or release workflows; they were removed (see git histor
 clean checkout: `yarn release:version <version>`, `yarn release:build` (builds each target's
 executable and addon into `release-artifacts/<target>/`), `node scripts/assemble-release.mjs
 release-artifacts release`, then `yarn release:verify:publish`. The publish verifier checks that
-`FREELLAMA_RELEASE_TAG`, when set, matches the workspace version.
+`FREELLAMA_RELEASE_TAG`, when set, matches the workspace version. It also requires every addon and
+executable to embed `freellama-source-commit=<sha>` (written by `packages/rust-core/build.rs`; also
+shown by `freellama --version`) equal to `git rev-parse HEAD`, and a clean tracked tree; a stale or
+`-dirty` artifact fails naming the file and the commit it was built from. CI checkouts without git
+set `FREELLAMA_RELEASE_COMMIT` to the 40-character release SHA. Then run `yarn release:smoke`: it packs
+`packages/cli`, `packages/mcp` and this host's `packages/native/<id>`, installs them into a fresh
+project and checks `freellama --version`, that `freellama --help` lists every subcommand of
+`target/release/freellama`, that the installed addon exports exactly the functions in
+`packages/mcp/native/index.d.ts`, and that the installed MCP server answers `tools/list` (needs the
+registry for the MCP dependencies). `yarn verify:production` runs both.
 Live E2E and hardware qualification remain separate operator checks; verify those receipts before tagging.
 
-Attach `release/SHA256SUMS` and binaries to the release. Publish the eight `@octocodeai/freellama-native-*` packages
+Attach `release/SHA256SUMS` and binaries to the release. Publish every `@octocodeai/freellama-native-*` package
 first, followed by the exact-version `@octocodeai/freellama` and `@octocodeai/freellama-mcp-server` packages. Publish
 the Rust crates from the same commit with `cargo publish --workspace` (it uploads `freellama-core`
 before `freellama-cli`; run it with `--dry-run` first). Configure registry credentials (`npm login`,
@@ -127,16 +136,15 @@ and require the same visible values before admitting work.
 Generate a token without placing the secret in shell history or the process list:
 
 ```bash
-freellama auth-token --out ~/.local/share/freellama/auth.token
+freellama auth-token --out ~/.octocode/freellama/auth.token
 ```
 
 The command creates a new file with mode `0600` on Unix and refuses to overwrite an existing
-token. Start the service with persistent feedback and authentication:
+token. Start the service with authentication:
 
 ```bash
 freellama serve \
-  --feedback-file ~/.local/share/freellama/feedback.json \
-  --auth-token-file ~/.local/share/freellama/auth.token \
+  --auth-token-file ~/.octocode/freellama/auth.token \
   --upstream http://127.0.0.1:11434 \
   --cpu-upstream http://127.0.0.1:11436 \
   --cpu-model nomic-embed-text:latest
@@ -147,10 +155,65 @@ adapters. Authentication covers both `/_freellama/v1/*` and Ollama-compatible pa
 Use `--allow-remote` only with an authentication token. The server refuses unauthenticated
 nonloopback listeners.
 
-The feedback snapshot is versioned, bounded by backend and task type, and replaced atomically.
-Only physically verified warm samples are saved. A corrupt or unsupported snapshot makes startup
-fail instead of silently discarding routing evidence. Use `--ephemeral-feedback` only for
-disposable tests.
+### Client keys
+
+Give each agent, team or tool its own key instead of sharing the operator token. Create one token
+file per client with `freellama auth-token --out`, then list them in `clients.toml` in the home (or
+pass `--client-keys` / `FREELLAMA_CLIENT_KEYS`):
+
+```toml
+[[client]]
+name = "octocode"
+token_file = "keys/octocode.token"   # relative to this file; mode 0600
+daily_tokens = 2000000               # optional; unmetered when absent
+admin = false                        # optional; true allows model files and config reload
+```
+
+A key's usage is the tokens the backend processed for it per UTC day: output plus the prompt tokens
+not reused from a prompt cache, for managed tasks and raw Ollama or OpenAI calls. The budget is
+checked before each generating request, so the request that crosses it finishes; later ones get 429
+with `code: "client_budget_exhausted"` and a `Retry-After` until the next UTC day. Reads still work.
+Non-admin keys get 403 (`client_not_admin`) for pull, push, copy, create, delete, blob uploads and
+`config/reload`. `usage` shows each key's `tokens_today`, `daily_tokens` and `remaining_today`; a
+client key sees only its own row. Today's totals start from the shared database at startup and
+then count this process's traffic. Point a client's `FREELLAMA_AUTH_TOKEN_FILE` at its own key. A
+serve that the MCP server starts itself treats that file as the operator token, so run `serve`
+yourself when you use client keys.
+
+### Other OpenAI-compatible engines
+
+`upstreams.toml` in the home (or `--upstreams` / `FREELLAMA_UPSTREAMS`) puts other engines behind
+the same endpoint:
+
+```toml
+[[upstream]]
+name = "lmstudio"
+url = "http://127.0.0.1:1234/v1"
+
+[[upstream]]
+name = "cloud"
+url = "https://api.example.com/v1"
+api_key_file = "keys/cloud.key"      # mode 0600; sent only over HTTPS or to this machine
+```
+
+Only OpenAI-compatible paths (`/v1/...`) route by model: `name/model` goes to that engine as
+`model`; a bare name goes to Ollama when Ollama has it, else to the first engine whose `/v1/models`
+lists it. `GET /v1/models` merges every list and names unreachable engines. A request to an engine is
+sent once (no retries), streams through unchanged, carries the `x-freellama-upstream` header and is
+counted in usage. An engine on this machine waits out host holds as a loaded Ollama model would; a
+remote one does not. Managed tasks, residency and the memory budget cover Ollama only, and there is no
+automatic failover from one engine to another.
+
+Everything the server learns lives in one `SQLite` database, `~/.octocode/freellama/freellama.sqlite`
+(`--home` or `FREELLAMA_HOME` moves the home): measured runner sizes, placement feedback, the usage
+ledger and the residency audit. It uses write-ahead logging, so several FreeLlama processes on one
+machine share it safely and learn from each other. The feedback snapshot is versioned and bounded by
+backend and task type; only physically verified warm samples are saved, and a schema-1 snapshot (0.7.x)
+is migrated on load (aggregates kept as diagnostics, original kept as `placement_feedback_v1_backup`);
+a corrupt or unknown-version snapshot starts empty with a stderr warning and `/health`
+`feedback.persistence.last_error`, and is never overwritten. A database file with
+another schema version is refused with its path; move or delete it to start a new one. Use
+`--ephemeral` only for disposable tests.
 
 ## Bound conversation history and residency
 
@@ -193,7 +256,7 @@ longer resident before returning. Inspect both `execution.observation` and
 
 Require these health contracts before admitting traffic:
 
-- `placement_feedback_persistence: versioned_atomic_snapshot_v1`
+- `placement_feedback_persistence: shared_database_snapshot_v2`
 - `authentication: optional_bearer_all_routes`
 - `immediate_unload_observation: observe_then_unload`
 - `placement_observation: ollama_api_ps_after_execution`

package/docs/README.md

@@ -2,7 +2,7 @@
 
 FreeLlama helps agents offload bounded work to local models through Ollama. The calling agent owns
 the task and verifies the answer; FreeLlama manages model selection and resources around execution.
-Start with the [repository MCP setup](https://github.com/bgauryy/FreeLlama/blob/main/README.md#mcp-server) to run one managed task.
+Start with the [repository README](https://github.com/bgauryy/FreeLlama/blob/main/README.md#install) to install and run one managed task.
 
 ## Choose a guide
 
@@ -11,8 +11,17 @@ Start with the [repository MCP setup](https://github.com/bgauryy/FreeLlama/blob/
 | Connect an agent and choose a tool | [MCP server](https://github.com/bgauryy/FreeLlama/blob/main/packages/mcp/README.md) |
 | Follow the agent's operating workflow | [FreeLlama skill](https://github.com/bgauryy/FreeLlama/blob/main/skills/freellama/SKILL.md) |
 | Look up commands and task controls | [CLI reference](CLI.md) |
+| Choose a model for a job | `models {view:"catalog"}`, `freellama catalog`, or the [local model catalog](LOCAL_MODELS.md) |
+| Follow a recipe for one job | [Cookbooks](cookbooks/README.md) |
+| Teach an agent the Octocode CLI in under 30 lines | [Local tools](octocode/LOCAL.md), [GitHub tools](octocode/GITHUB.md) |
 | Understand task and resource ownership | [Architecture](ARCHITECTURE.md) |
+| See every feature, routing and admission in depth, and why FreeLlama exists | [Features](FEATURES.md) |
+| Compare FreeLlama with Ollama, llama.cpp, llama-swap and LocalAI | [Comparison](COMPARISON.md) |
+| See supported model forms and dated measured results | [Model forms and results](MODEL_FORMS_AND_RESULTS.md) |
+| Reduce reloads, prompt cost and memory; tune Ollama settings | [Tuning](TUNING.md) |
+| Build from source; read operating notes (response store, port 11435, certificates) | [Build and notes](BUILD_AND_NOTES.md) |
 | Choose a model and interpret its metadata | [Model selection](MODEL_SELECTION.md), [model metadata](MODEL_SELECTION.md#model-metadata) |
+| Inspect quantization, placement, or neuron activations | [Model introspection](MODEL_INTROSPECTION.md) |
 | Configure CPU helpers alongside GPU work | [CPU/GPU routing](CPU_GPU_ROUTING.md) |
 | Inspect queues, memory, usage, and live settings | [Monitoring](MONITORING.md) |
 | Retain history, affinity, or residency | [Scopes and warming](SCOPES_AND_WARMING.md) |
@@ -22,11 +31,7 @@ Start with the [repository MCP setup](https://github.com/bgauryy/FreeLlama/blob/
 | Run development and release checks | [Testing](TESTING.md), [release packages](TESTING.md#verify-release-packages) |
 | Embed the routing core | [Rust core](https://github.com/bgauryy/FreeLlama/blob/main/packages/rust-core/README.md) |
 | Use the read-only status page | `/_freellama/ui` on a running `serve` (see [Monitoring](MONITORING.md)) |
-
-## Evidence and design
-
-| Goal | Source |
-|---|---|
+| Configure durable receipts, idempotency and bounded job retention | [Deferred jobs and recovery](DEFERRED_JOBS.md) |
 
 Historical measurements describe one model, machine, and workload. Inspect current state and
 repeat the relevant checks before using them to select models or set resource limits.

package/docs/SCOPES_AND_WARMING.md

@@ -145,9 +145,14 @@ Expired entries are removed during store operations.
 
 ## Warm API
 
-`POST /warm`, MCP `run_task {warm:true}`, and CLI `warm` load an exact installed model with a matching
-non-embedding task profile. They do not accept prompts or messages, pull a model, or store conversation
-history. The default task profile is `completion`.
+`POST /warm`, MCP `ollama_manage {action:"warm"}`, and CLI `warm` load an exact installed model with a matching
+task profile; `task: "embedding"` loads an embedding model with an empty `/api/embed` request. A
+chat warm may carry `messages` (MCP: `systemPrompt` and/or `messages`): the runner evaluates them
+into its prompt cache with `num_predict: 0` and generates nothing, so a later request that starts
+with the same messages, at the same context, reuses that work (its receipt reports
+`cached_prompt_tokens`). Warming never pulls a model or stores conversation history. The default task
+profile is `completion`. Ollama has no load-only request for decision models: they load on their
+first `/v1/systemone` call, and `keep_alive` on that call sets their residency.
 
 | HTTP field | MCP field | Meaning |
 |---|---|---|
@@ -161,6 +166,13 @@ history. The default task profile is `completion`.
 | `max_wait_seconds` | `maxWaitSeconds` | Shorter admission/resource wait budget. |
 | `timeout_seconds` | `timeoutSeconds` | Total request deadline, capped by server policy. |
 | `defer` | `defer` | Return a job handle for the same managed operation. |
+| `messages` | `systemPrompt`, `messages` | A prompt prefix to evaluate into the runner's prompt cache. |
+
+A warm of a model that is not resident is a load. Like any load it takes the backend's load lock
+(see [Load lock and use lease](CPU_GPU_ROUTING.md#load-lock-and-use-lease)) until its runner is
+observed in `/api/ps`, so it queues behind another load on the same backend but never waits for
+another model's decoding. It makes room only by unloading idle runners: a model with a running
+task is never evicted for a warm.
 
 Synchronous warming returns HTTP `200`; deferred warming returns HTTP `202`. Inspect deferred
 work through `task_jobs` or `jobs` using its returned ID. Warming reuses ordinary routing,
@@ -177,9 +189,17 @@ chat requires Ollama's `done:true`; an absent or non-boolean completion marker i
 These are separate observations in Ollama's [chat response](https://docs.ollama.com/api/chat) and
 [running-model inventory](https://docs.ollama.com/api/ps).
 
-With `keep_alive:"0"`, warming observes placement and then unloads. The `warm` receipt describes
-residency after unload: `loaded:false` on verified unload, with
-`residency_source:"ollama_api_ps_after_unload"`.
+With `keep_alive:"0"` on a model that is not resident, warming loads it, observes placement and then
+unloads. The `warm` receipt describes residency after unload: `loaded:false` on verified unload,
+with `residency_source:"ollama_api_ps_after_unload"`. On a model that is already resident,
+`keep_alive:"0"` is a plain unload: it waits for that model's running tasks, unloads it and returns
+`{"unloaded": true, "status": "unloaded", "unload_ms": …}` without loading or routing anything.
+`POST /_freellama/v1/unload {"model": …}` (CLI `unload`) does the same and never loads a model that is
+not resident (`status: "not_resident"`).
+
+A warm without `task` loads an embedding-only model as an embedding task, so
+`{"model":"nomic-embed-text:v1.5"}` works without `"task":"embedding"`. Warms and preloads teach
+FreeLlama a model's load time but do not count as uses, so they never lengthen their own residency.
 
 Warming uses the same truncation and context-shift policy as managed chat. This preserves a
 compatible runner profile instead of forcing a transition solely because warming omitted those
@@ -187,6 +207,34 @@ controls. Use a context/task profile compatible with the following task. Warming
 require another runner transition. Warm residency does not reserve a later execution slot. Cache reuse depends on the compatible
 profile, unchanged prefix, and inference backend. See [Managed execution](ARCHITECTURE.md#managed-task-execution).
 
+## Preload
+
+The runtime file's `[preload]` table loads models without a caller asking:
+
+- `startup`: tags, or tables with `model` and optional `context_tokens`, `keep_alive` (default: the
+  warming policy's `max_seconds`) and `messages`, loaded in order once the server is up and again
+  for entries a reloaded runtime file adds. `FREELLAMA_PRELOAD` sets a comma-separated list.
+- `predictive` (on by default): every switch from one model to another within
+  `follow_window_seconds` (600) is counted in the shared database. Once at least `min_observations`
+  (3) switches away from a model happened and `min_probability` (0.5) of them went to one successor,
+  finishing a caller's work on the model loads that successor in the background, but only when it
+  is not loaded, fits without unloading anything (budget, GPU and host memory), the host is under no
+  pressure, and nothing is running or queued on the backend. Only the primary backend predicts.
+  `FREELLAMA_PREDICTIVE_PRELOAD=0` turns it off.
+
+  Predictive warm is refused on a discrete-GPU or remote backend. Vendor-wide free VRAM does not
+  say which devices that Ollama process may allocate on, and nothing in FreeLlama verifies it
+  (`gpu_capacity_scope` stays `host_aggregate_not_verified_for_backend`), so a speculative load
+  could displace work on a pool it cannot see. The skip reason is `GPU device eligibility for this
+  Ollama backend is unverified` (`opportunistic_warm_skipped`). Unified-memory hosts and the CPU
+  backend, where host memory is the budget, still predict. Explicit warms, startup preloads and
+  managed tasks on a discrete GPU run through ordinary admission.
+
+Each preload is a background warm under the usual admission, memory and pressure rules, and the
+residency audit records it as `preload` (or `preload_failed`) with trigger `startup` or
+`predicted_after:<model>`. Switches are counted per process from the order requests finish, so
+several agents interleaving models add noise that the probability threshold has to outweigh.
+
 For a bounded warm operation, replace `MODEL_TAG` with an exact installed tag:
 
 ```bash
@@ -208,7 +256,7 @@ Matching environment values override `[warming]` file values.
 |---|---|---:|---|
 | `min_seconds` | `FREELLAMA_KEEP_ALIVE_MIN_SECONDS` | `15` | Minimum automatic duration. |
 | `max_seconds` | `FREELLAMA_KEEP_ALIVE_MAX_SECONDS` | `900` | Maximum automatic duration. |
-| `base_seconds` | `FREELLAMA_KEEP_ALIVE_BASE_SECONDS` | `60` | Base duration before measured reuse and load cost. |
+| `base_seconds` | `FREELLAMA_KEEP_ALIVE_BASE_SECONDS` | `300` | Base duration before measured reuse and load cost; matches Ollama's five-minute default. |
 | `reuse_gain_seconds` | `FREELLAMA_KEEP_ALIVE_REUSE_GAIN_SECONDS` | `30` | Additional duration per recent use. |
 | `load_multiplier` | `FREELLAMA_KEEP_ALIVE_LOAD_MULTIPLIER` | `2.0` | Multiplier for measured load seconds. |
 | `pressure_factor` | `FREELLAMA_KEEP_ALIVE_PRESSURE_FACTOR` | `0.25` | Duration factor while observed resource pressure holds admission. |

package/docs/TESTING.md

@@ -80,6 +80,14 @@ cargo clippy --all-targets -- -D warnings
 
 Routing regressions run in the default Rust suite; none are hidden behind `#[ignore]`. Add a
 focused contract test for each repaired behavior and keep it enabled in ordinary release checks.
+Completion-planning regressions cover payload fit before ranking, global quality gates across
+backends, exact-profile history retention, idempotent snapshot merges, unknown phase
+costs, prefill/decode tradeoffs and cached-token normalization. Runner tests reject mismatched
+digests/contexts and distinguish reported setup duration from an observed load transition.
+Resource tests require released forecasts to invalidate both cached and already-returned host
+snapshots. Predictive warm fixtures use a mock Ollama to assert no queuing or eviction under
+contention; they require permission to bind a loopback listener. MCP tests require accepted jobs
+to retain their identity and fail without re-execution if result retention fails.
 Resource contracts cover conservative full host-memory reservations when discrete-GPU telemetry is
 missing, and immediate-unload cleanup after a failed inference response body. The latter also checks
 that uncertain inference is never replayed and ordinary keep-alive requests retain their runner.
@@ -198,13 +206,27 @@ placement, shared-memory contention, or OCR quality.
 The historical CPU/GPU observations above predate the corrected paired-run acceptance gate.
 They remain records of those runs and do not qualify a release.
 
+## Run live model harnesses
+
+These scripts measure installed models through a running `serve`. They are not part of any test
+tier, and each writes a dated report to the git-ignored `.octocode/local-test/`.
+
+| Script | Measures |
+|---|---|
+| `scripts/local-test/rivals.mjs` | The graded questions in `cases.mjs` on rival models per job, three rounds, resumable |
+| `scripts/local-test/lifecycle.mjs` | Cold and warm calls, keep-alive, unload, switching, queue priorities, prompt caching, CPU vs GPU |
+| `scripts/local-test/assist.mjs` | Issue to code, duplicates, triage and OCR on a real repository |
+| `scripts/local-test/media.mjs` | Speech round trip, multimodal image search, embedding comparison through MCP |
+| `scripts/local-test/octocode-agent.mjs` | A local model driving Octocode through `delegate_research` |
+
 ## Verify release packages
 
 ```bash
 # Collect each target's `freellama` executable and `freellama.<target>.node`
-# under release-artifacts/<target>/, then stage all eight platform packages.
+# under release-artifacts/<target>/, then stage every platform package in scripts/release-platforms.mjs.
 node scripts/assemble-release.mjs release-artifacts release
-yarn release:verify:publish
+yarn release:verify:publish   # also requires each artifact's embedded commit == HEAD, clean tree
+yarn release:smoke            # packed-install smoke of the host platform package
 ```
 
 Build every claimed CLI/native target in a clean environment, assemble `SHA256SUMS`, run the full