@octocodeai/freellama-mcp-server 0.1.0

package/docs/INDEX.md

@@ -0,0 +1,15 @@
+# FreeLlama packaged documentation
+
+Read only the document relevant to the current operation. Repository `docs/` is the source of truth.
+
+- [ADR: Resource-aware backend routing](ADR_RESOURCE_AWARE_BACKEND_ROUTING.md)
+- [FreeLlama architecture](ARCHITECTURE.md)
+- [CLI reference](CLI.md)
+- [Run models on CPU and GPU](CPU_GPU_ROUTING.md)
+- [Token economics](ECONOMICS.md)
+- [Choose local models](MODEL_SELECTION.md)
+- [Ollama sidecar boundary](OLLAMA_SIDECAR.md)
+- [Ollama and FreeLlama optimization boundary](OLLAMA_SYSTEM_OPTIMIZATION.md)
+- [Prepare FreeLlama for production](PRODUCTION.md)
+- [Product positioning](PRODUCT_POSITIONING.md)
+- [Test FreeLlama](TESTING.md)

package/docs/MODEL_SELECTION.md

@@ -0,0 +1,65 @@
+# Choose local models
+
+Choose models from local evidence, not model-card popularity. The measurements below come from one
+Apple M4 Pro machine with 48 GB unified memory and are not compiled into routing. Re-run
+`npx @octocodeai/freellama doctor`, `models`, `bench-all`, and your quality suite on the target hardware.
+`doctor.machine.memory_bytes` is host RAM; a discrete GPU's usable VRAM must be observed separately.
+
+```mermaid
+flowchart TD
+    T{"What must the task do?"}
+    T -->|"Generate text, code, or inspect images"| L["Start with one qualified large multimodal model"]
+    T -->|"Create retrieval vectors"| E["Use a small embedding model"]
+    L --> Q{"Passes held-out quality checks?"}
+    E --> Q
+    Q -->|"No"| N["Reject or evaluate another model"]
+    Q -->|"Yes"| M{"Fits with workload concurrency?"}
+    M -->|"No"| N
+    M -->|"Yes"| B["Benchmark locally and add policy evidence"]
+```
+
+## Text, code, and vision
+
+`qwen3.8:27b-mlx` was the strongest general model in the local evaluations. It handled coding,
+grounded research, image description, OCR, and summarization. Using one large multimodal model also
+avoided contention between separate heavy text and vision runners.
+
+`muse-glimmer:30b-mlx` was a credible alternative and answered 29 of 30 questions correctly
+(96.7%) with the Bash adapter, but it was slower and did not add a required capability in this
+workload.
+
+Do not assume small research models are adequate. The local grounded-lookup sample fell to 2/8 at
+7B, 3/8 at 3B, and 0/8 at 0.5B. `delegate_research` can refuse models whose runtime evidence marks
+them unusable rather than spending tokens on a known-poor route.
+
+## Embeddings
+
+`nomic-embed-text` matched the highest recall with the lowest measured index time and model size in
+the repository retrieval sample:
+
+| Model | Recall at 3 | Index time | Dimensions | Size |
+|---|---:|---:|---:|---:|
+| `nomic-embed-text` | 5/6 | 4.2 seconds | 768 | 274 MB |
+| `embeddinggemma:300m` | 5/6 | 4.9 seconds | 768 | 622 MB |
+| `qwen3-embedding:0.6b` | 4/6 | 14.8 seconds | 1,024 | 639 MB |
+
+Use embeddings for semantic similarity, grouping, deduplication, and classification. For exact code
+keywords, repository search was both faster and more accurate in the measured sample.
+
+## Place helper models on CPU when useful
+
+A small embedding or intent model can be assigned to a second CPU Ollama process while a large
+generation model remains resident on the primary GPU-capable process. This can overlap independent
+work, but it does not make CPU inference fast. Apple silicon shares memory bandwidth; discrete-GPU
+systems have different transfer and contention costs and need their own benchmark.
+
+The measured warmed embedding-plus-completion workload improved from a 37.997-second sequential
+median to a 28.233-second parallel median, a 1.346-times speedup. Treat that result as evidence for
+this machine and workload, not a universal model-placement rule. Follow
+[CPU and GPU model routing](CPU_GPU_ROUTING.md) to reproduce it.
+
+## Turn measurements into routing evidence
+
+`bench-all` measures local performance. A quality benchmark measures correctness. FreeLlama needs
+both the generated task policy and local benchmark data before grading a route medium confidence.
+See the [CLI reference](CLI.md#earn-medium-routing-confidence) for the commands.

package/docs/OLLAMA_SIDECAR.md

@@ -0,0 +1,76 @@
+# Ollama sidecar boundary
+
+FreeLlama integrates with Ollama as a loopback HTTP sidecar, not an in-process inference plugin.
+Ollama owns model execution; FreeLlama owns policy and coordination around it.
+
+```mermaid
+flowchart LR
+    C["Ollama-compatible client"] --> F["FreeLlama loopback sidecar"]
+    F -->|"raw /api and /v1"| G["Primary Ollama"]
+    F -->|"managed primary task"| G
+    F -->|"managed assigned task"| U["Optional CPU Ollama"]
+    G --> E["Metal, MLX, CUDA, ROCm, Vulkan, or CPU runner"]
+    U --> E2["CPU runner"]
+```
+
+This boundary avoids an Ollama fork and preserves its documented HTTP contract. It also makes a
+second, process-isolated CPU backend possible without changing arbitrary client payloads.
+
+## Divide ownership
+
+| Problem | Owner |
+|---|---|
+| Kernels, prompt processing, and token generation | Ollama and its runner dependencies |
+| Runner loading, eviction, and scheduling within one server | Ollama |
+| Prompt templates and model defaults | Ollama model metadata or a `Modelfile` |
+| Capability and memory filtering | FreeLlama managed routes |
+| Evidence policy, confidence, and route receipts | FreeLlama managed routes |
+| Session affinity, per-backend bounded admission, and warm runtime feedback | FreeLlama managed routes |
+| Exact model assignment to an optional CPU process | FreeLlama configuration |
+| Guarded CPU/GPU preference within assigned eligible models | FreeLlama managed routes |
+| Authentication and remote exposure | FreeLlama bearer middleware; external ingress owns TLS and tenant authorization |
+
+FreeLlama does not make an individual model decode faster. It can improve workload completion by
+avoiding unsuitable routes, reducing avoidable transitions, refusing unsupported work early, and
+overlapping independent CPU and GPU model requests when the machine benefits from that layout.
+
+## Preserve compatibility
+
+`packages/rust-core/src/proxy.rs` forwards paths, queries, request bodies, and streaming response
+bodies. It removes hop-by-hop headers and adds `x-freellama-proxy: 1` to responses. Raw `/api/*`
+and `/v1/*` requests always target the primary upstream and are not rewritten for CPU placement.
+
+```mermaid
+flowchart TD
+    R["Incoming request"] --> K{"Managed /_freellama/v1 route?"}
+    K -->|"No"| P["Stream unchanged to primary Ollama"]
+    K -->|"Yes"| V["Validate, route, assign backend, and admit"]
+    V --> X["Build bounded Ollama request"]
+    X --> O["Execute on assigned upstream"]
+```
+
+The distinction is intentional: existing Ollama clients retain predictable compatibility, while
+clients that opt into the managed API receive routing and policy behavior.
+
+## Know the remaining limits
+
+Native passthrough traffic remains under Ollama's scheduler and does not participate in FreeLlama's
+transition locks. The managed sidecar also does not enforce live memory-pressure or thermal
+admission. A configured but unreachable CPU backend makes managed catalog discovery fail closed;
+raw primary passthrough remains usable.
+
+Runtime feedback is task-specific, token-normalized, and warm-only. It can steer
+unpinned `fastest` or `balanced` work after three samples on each backend and a greater-than-10%
+advantage; it cannot change CPU eligibility or steer quality routing. The CLI persists a bounded,
+versioned snapshot atomically by default, so verified evidence survives restart without changing
+Ollama residency.
+
+Read [Architecture](ARCHITECTURE.md) for all managed flows,
+[CPU and GPU model routing](CPU_GPU_ROUTING.md) for the two-process procedure, and
+[Ollama and FreeLlama optimization](OLLAMA_SYSTEM_OPTIMIZATION.md) for the full tuning boundary.
+
+## Sources
+
+- [Ollama API introduction](https://docs.ollama.com/api/introduction)
+- [Ollama streaming behavior](https://docs.ollama.com/api/streaming)
+- [Ollama usage metrics](https://docs.ollama.com/api/usage)

package/docs/OLLAMA_SYSTEM_OPTIMIZATION.md

@@ -0,0 +1,254 @@
+# Ollama and FreeLlama optimization boundary
+
+Ollama already owns inference optimization. FreeLlama can improve completed
+work per hour by choosing an eligible model and avoiding unnecessary model
+changes, but it does not make an individual model decode faster.
+
+This page explains the active request flow, the settings Ollama owns, the
+current machine audit, and the optimization work that remains.
+
+The architecture is portable across Ollama's Metal, CUDA, ROCm, Vulkan, and CPU paths. The audit
+later on this page is intentionally Mac-specific. FreeLlama discovers host RAM per operating system
+and keeps device selection in each Ollama process; it does not translate one machine's memory tier,
+core count, or benchmark into another machine's defaults.
+
+## Decision
+
+Use stock Ollama as the inference runtime. Keep FreeLlama as an optional Rust
+control plane for evidence-qualified routing, bounded request profiles, session
+affinity, and policy enforcement.
+
+Do not advertise FreeLlama as an accelerator or an Ollama inference plugin. It does not tune
+kernels, split one model's layers across CPU and GPU, manage macOS memory pressure, or improve that
+model's tokens per second. It can assign different models to isolated CPU and GPU-capable Ollama
+processes, allowing independent workloads to overlap.
+
+## Request flow
+
+FreeLlama has four distinct paths:
+
+| Request | FreeLlama work | Ollama work | Result |
+|---|---|---|---|
+| Native `/api/*` or `/v1/*` | Streams bytes to the primary compatibility upstream | Performs normal inference and scheduling | Exact Ollama response |
+| `POST /_freellama/v1/routes` | Discovers models, filters capabilities, and applies policy | Supplies installed-model and residency data | Model and request profile; no inference |
+| `POST /_freellama/v1/natural-routes` | Normalizes a schema-bound intent, then runs the deterministic router | Runs the small intent model | Model and request profile; selected task does not run |
+| `POST /_freellama/v1/tasks` | Selects a route and backend, applies managed-task admission, and constructs one bounded request | Runs one nonstreaming chat or embedding request on the assigned process | Route receipt, placement, admission mode, prompt-free metrics, and Ollama response |
+
+The natural-language path is intentionally two-stage. The small model cannot
+name a final model. It returns only a task, objective, context hint, and tool or
+vision requirements. Deterministic guards correct explicit requirements before
+the ordinary router selects a model.
+
+Calling `natural-routes` does not run the selected task. A client must use the
+returned profile or submit a structured `tasks` request. There is no atomic
+natural-language route-and-run endpoint in the current release.
+
+Chat and embedding tasks forward route options. A captured-upstream contract
+test verifies embedding option parity.
+
+## What Ollama already optimizes
+
+Across supported backends, Ollama provides the following behavior without FreeLlama:
+
+- The runtime selects supported acceleration such as Metal, CUDA, ROCm, or Vulkan from the host and
+  its process-level device visibility.
+- The scheduler fits model, context, and compute allocations to available
+  device memory.
+- The runner selects CPU threads and GPU execution; applications do not need to
+  divide one inference manually between the CPU and GPU.
+- Ollama manages request queues plus model loading and eviction. It supports
+  same-model parallel requests when memory permits.
+- Ollama reports token counts plus load, prompt-processing, and generation
+  durations in API responses.
+- Ollama supports request-specific context length and model residency through
+  `num_ctx` and `keep_alive`.
+
+Ollama's documented concurrency controls are global server settings. Increasing
+`OLLAMA_NUM_PARALLEL` multiplies context memory by the parallel-request count.
+Increasing `OLLAMA_MAX_LOADED_MODELS` does not create memory; models still need
+to fit. Use `1` for both values on an unmeasured deployment, then increase one variable at a time
+only when a representative concurrent benchmark improves completed tasks per hour without memory
+pressure.
+
+Keep three signals distinct when operating FreeLlama: `context_window_fit` only compares requested
+context with advertised model context; FreeLlama's K/V preflight is a conservative metadata lower
+bound; and a completed runner is the only placement observation. None is live free-memory or
+throughput proof. `doctor {view:"scheduler"}` reports configuration and a point-in-time admission
+snapshot, not a fairness or parallel-decoding benchmark.
+
+FreeLlama's admission queue and Ollama's internal queue solve different problems. FreeLlama limits
+weighted managed work per backend and applies `--max-queue-wait-seconds`; `OLLAMA_MAX_QUEUE` bounds
+requests that an Ollama process has accepted but cannot schedule. Raw compatibility traffic skips
+FreeLlama admission and reaches the primary Ollama queue directly.
+
+Ollama request context and research-adapter context are also separate budgets. Managed tasks set
+request-specific `num_ctx` from the selected task profile. The coding adapters fit their tool
+conversation inside `FREELLAMA_AGENT_NUM_CTX`: by default, they preserve the system prompt and
+original question, compact the oldest observations first, retain recent evidence, and fail before
+the Ollama call if pinned content cannot fit. Raising the Ollama server's global context does not
+disable adapter compaction or increase its configured budget. See [Agent adapters](../AGENTS.md)
+for the complete pagination and compaction contract.
+
+## Current Mac audit
+
+This audit captures the following snapshot from September 1, 2026. Treat it as machine
+evidence, not a portable default.
+
+| Item | Observed value | Interpretation |
+|---|---|---|
+| Hardware | Apple M4 Pro, 14 CPU cores, 48 GB unified memory | CPU and GPU share the memory system |
+| macOS | 15.7.2 | Native Ollama supports this Apple-silicon system |
+| Ollama server | 0.33.2 | Active application server on `127.0.0.1:11434`; re-read with `doctor` rather than trusting this row |
+| First CLI on `PATH` | `/usr/local/bin/ollama`, 0.33.2 | Matches the active server |
+| Metal device budget | 36 GiB reported by Ollama | This is the scheduler's observed device budget, not all 48 GB |
+| Default context | 32,768 tokens | Ollama selected its 24–48 GiB device-memory tier |
+| Parallel requests | 1 | Appropriate baseline for interactive and large-model tests |
+| Keep-alive | 5 minutes | Ollama default |
+| Loaded-model limit | `OLLAMA_MAX_LOADED_MODELS=2` | Explicit service configuration for the measured topology |
+| K/V-cache type | `OLLAMA_KV_CACHE_TYPE=q8_0` | Explicit memory-saving configuration; requalify quality per important model |
+| Flash Attention | `auto` | Ollama enables it automatically on supported backends; force `1` only after a measured compatibility check. Required for quantized K/V cache. |
+| GGUF CPU threads | 10 worker threads | Ollama selected the 10 performance cores rather than all 14 cores |
+| MLX | GPU runner observed | MLX models use their own runner path, not the GGUF tuning path |
+| Power mode | Automatic on battery and AC power | High Power Mode remains an unmeasured experiment |
+
+The server log also showed Metal fusion, concurrent Metal execution, graph
+optimization, and automatic parameter fitting. FreeLlama must not claim these
+as its own optimizations.
+
+### Verify CLI and server alignment
+
+The selected CLI is `/usr/local/bin/ollama`, and both the CLI and active server report 0.33.2.
+Verify this alignment after changing Ollama installations or service definitions:
+
+```bash
+command -v ollama
+ollama --version
+curl --silent http://127.0.0.1:11434/api/version
+```
+
+The client and server versions must match before you test a CLI feature. A mismatch does not prove
+an inference slowdown, but it can produce incompatible commands, flags, or diagnostics.
+
+## What FreeLlama adds today
+
+The active Rust implementation adds:
+
+- capability filtering before speed ranking;
+- explicit `fastest`, `balanced`, and `quality` evidence contracts;
+- fail-closed quality routing when the policy has no qualified model;
+- task-specific context, output, thinking, and tool-validation profiles;
+- session affinity and preference for an eligible resident model;
+- separate transition locks for primary and CPU backends, with shared execution permits for
+  resident tasks and an exclusive permit for a cold transition;
+- exact-model placement on an optional second CPU Ollama process, including `num_gpu=0` for its
+  managed requests;
+- guarded agent placement preferences plus a normalized, three-warm-sample, 10%-advantage feedback
+  loop for `fastest` and `balanced` work;
+- independent 3:2:1 priority-fair weighted GPU and CPU admission pools, so a GPU burst cannot
+  starve a CPU helper or indefinitely starve background managed work;
+- prompt-free load, prompt-processing, and output-generation metrics derived
+  from Ollama's response fields;
+- a local natural-language intent interpreter that cannot choose a model;
+- an Ollama-compatible loopback endpoint and an escape hatch to port 11434.
+
+These features can reduce reloads or avoid an unsuitable model. They do not
+change the selected model's prompt-processing or decode rate.
+
+The measured dual-backend workload improved median completion time from 37.997 to 28.233 seconds
+(1.346 times, or 25.70%). Qwen reported 19,175,677,668 GPU-resident bytes, Nomic reported zero,
+and all requests carried correct backend receipts with zero FreeLlama queue wait. One of three
+parallel trials was slower than sequential, so this remains a workload-level optimization rather
+than an inference-speed claim.
+
+## Unsupported optimizations
+
+FreeLlama does not provide:
+
+- a live minimum-free-VRAM guard. FreeLlama does calculate a metadata-backed F16 K/V lower bound
+  and refuses a known impossible CPU/unified-memory floor, but Ollama remains the authority for
+  current free memory, runner graph allocation, and final loading;
+- transition coordination for native passthrough requests or processes that
+  bypass the managed `tasks` endpoint;
+- live thermal, power-mode, or swap-aware routing;
+- online token-rate, error-rate, or cold-load-duration scoring;
+- per-engine tuning for GGUF compared with MLX;
+- dynamic `keep_alive` or concurrency limits by workload;
+- an atomic natural-language route-and-run operation;
+- held-out quality policies for completion, coding, tools, browser, vision, or
+  long-context tasks in the example policy;
+- quality-aware or thermal-aware CPU/GPU load balancing. Automatic placement uses persisted,
+  prompt-free warm-latency aggregates only within exact operator-owned assignments.
+
+The RFC describes several of these capabilities as planned phases. Do not treat
+the RFC as evidence that they have shipped.
+
+## Optimization decisions
+
+Use this table before changing Ollama or FreeLlama:
+
+| Candidate change | Default decision | Reason |
+|---|---|---|
+| Expose Ollama beyond loopback | Do not add | Keep `OLLAMA_HOST` on loopback and put authenticated TLS ingress in front of FreeLlama instead |
+| Permit Ollama cloud features in a local-only deployment | Do not add | Set `OLLAMA_NO_CLOUD=1` when the deployment promises local-only operation |
+| Force layers of one model between CPU and GPU | Do not add | Ollama already fits and offloads the model; partial CPU fallback usually reduces decode throughput |
+| Assign different models to isolated CPU and GPU-capable processes | Supported, benchmark per workload | Process isolation is reliable, but CPU speed and shared-memory pressure still determine whether overlap helps |
+| Add macOS kernel or `sysctl` tuning | Do not add | Ollama does not document a supported, product-specific kernel setting. Unified-memory behavior belongs to macOS and Metal |
+| Run Ollama in Docker on macOS | Do not use for performance | Docker Desktop does not provide Ollama GPU acceleration on macOS |
+| Increase context globally | Do not add | Context memory grows with length; set the smallest sufficient `num_ctx` per request |
+| Set parallel requests above 1 | Benchmark first | It can improve concurrent throughput but multiplies context memory and can hurt interactive latency |
+| Keep more than one model loaded per process | Benchmark first | Model weights, runtime buffers, and every active context must fit together; start with `OLLAMA_MAX_LOADED_MODELS=1` |
+| Leave Ollama's queue without a deployment limit | Do not add | Set a measured `OLLAMA_MAX_QUEUE`; it remains separate from FreeLlama's admission deadline |
+| Pin multiple 18–21 GB models | Do not add by default | The observed 36 GiB Metal budget cannot safely establish that two heavy models plus contexts fit |
+| Force Flash Attention | Benchmark per engine and model | The inspected GGUF path already enabled it automatically; MLX has a different runner |
+| Use `q8_0` K/V cache | **8/10; qualify, then prefer for parallel/long-context work** | Ollama documents roughly half the KV memory of `f16` with very small precision loss; it is the practical companion to higher `OLLAMA_NUM_PARALLEL`, but it remains a process-wide quality tradeoff |
+| Use `q4_0` K/V cache | Reject as a default | It saves more memory with a larger possible quality loss |
+| Enable High Power Mode | Sustained-load experiment only | Apple documents higher sustained performance. Local token-rate and energy effects are not measured |
+| Keep the small intent model resident | Keep bounded | It reduces routing latency but consumes memory and can affect a large-model transition |
+
+## Measurement contract
+
+Measure optimization with Ollama's response fields:
+
+- prompt tokens per second = `prompt_eval_count / prompt_eval_duration` in
+  seconds;
+- output tokens per second = `eval_count / eval_duration` in seconds;
+- cold-load cost = `load_duration`;
+- end-to-end latency = `total_duration` plus measured FreeLlama overhead;
+- workload throughput = successful, quality-qualified tasks per hour.
+
+Compare one variable at a time against stock Ollama. Include cold and warm runs,
+fixed prompts, fixed output budgets, exact-output guardrails, quality guardrails, resident
+memory, swap, and failure counts. A higher token rate is not an improvement if
+quality falls, requests fail, or model transitions dominate the workload.
+
+Use `cargo run -- bench-all` (`packages/rust-core/src/model_bench.rs`) for model selection evidence.
+Use [CPU and GPU model routing](CPU_GPU_ROUTING.md) for the separate-process benchmark and
+verification procedure.
+
+- Local-model router RFC — lived in the gitignored `.octocode/` workspace and is no longer in the tree; the shipped design it argued for is [ARCHITECTURE.md](ARCHITECTURE.md)
+
+## Recommended next work
+
+1. Keep the CLI executable aligned with the active Ollama server; `doctor` reports the selected
+   path, both versions, and effective memory-related settings.
+2. Back up the versioned prompt-free feedback snapshot with the policy and deployed binary.
+   Verified warm work-unit latency and queue wait survive restart; raw prompts are never stored.
+3. Validate the metadata-backed K/V lower bound against a frozen alternating-model workload before
+   widening it beyond CPU/unified-memory impossible-floor refusal. Keep Ollama as final authority.
+4. Extend transition coordination only if managed streaming or another parsed
+   execution path needs it. Native passthrough remains Ollama-owned.
+5. Evaluate request-specific context and residency profiles before testing global K/V-cache or
+   concurrency changes. Keep Flash Attention automatic on supported backends or set it explicitly
+   to `1` before selecting a quantized K/V cache.
+6. Promote a hardware-and-engine profile only after `benchmark/hardware/run_validation.py` and its
+   held-out quality workload pass on a real self-hosted runner.
+
+## Sources
+
+- [Ollama FAQ and server settings](https://docs.ollama.com/faq)
+- [Ollama context length](https://docs.ollama.com/context-length)
+- [Ollama hardware and Metal support](https://docs.ollama.com/gpu)
+- [Ollama API usage metrics](https://docs.ollama.com/api/usage)
+- [Ollama embedding API](https://docs.ollama.com/api/embed)
+- [Ollama macOS support](https://docs.ollama.com/macos)
+- [Apple power modes](https://support.apple.com/en-lamr/101613)

package/docs/PRODUCTION.md

@@ -0,0 +1,195 @@
+# Prepare FreeLlama for production
+
+This runbook prepares a single-operator or trusted-team FreeLlama deployment. FreeLlama now
+supports authenticated nonloopback listeners, but it is not a multi-tenant authorization or
+billing service. Put TLS and tenant-specific rate limits at an external ingress when traffic
+crosses a machine boundary.
+
+## Meet the prerequisites
+
+- Install one Ollama CLI and server version. Run `freellama doctor` and resolve any mismatch.
+- Install Node.js 20 or later for MCP and npm distribution use.
+- Use the release artifacts for your operating system, or build with Rust 1.85 or later.
+- Preinstall and benchmark exact model tags. Discovery never authorizes a pull.
+- Keep the primary and optional CPU Ollama processes on distinct loopback sockets.
+
+After a tag has been published, install that exact checksummed release on macOS or Linux:
+
+```bash
+curl -fsSLO https://raw.githubusercontent.com/bgauryy/FreeLlama/main/scripts/install.sh
+sh install.sh --version vX.Y.Z
+```
+
+The installer downloads the exact-version binary and verifies it against the release's
+`SHA256SUMS`. From a Windows checkout, run the following command after replacing `VERSION` with the
+published tag:
+
+```powershell
+powershell -ExecutionPolicy Bypass -File .\scripts\install.ps1 -Version VERSION
+```
+
+There is no repository-hosted release automation. Build and test every claimed target in a clean
+environment, place each executable and matching addon under `release-artifacts/<target>/`, run
+`yarn release:assemble release-artifacts release` and `yarn release:verify:publish`, then attach
+`release/SHA256SUMS` and binaries to the release. Publish the eight `@octocodeai/freellama-native-*` packages
+first, followed by the exact-version `@octocodeai/freellama` and `@octocodeai/freellama-mcp-server` packages. Configure
+registry credentials in the release environment; never store them in the repository.
+
+## Configure Ollama explicitly
+
+Start with one model and one decode stream per Ollama process. Increase either value only after a
+representative workload proves that the additional weights, contexts, and K/V caches fit without
+swap, eviction churn, or unacceptable queueing.
+
+| Setting | Starting value | Reason |
+|---|---:|---|
+| `OLLAMA_HOST` | Primary `127.0.0.1:11434`; CPU `127.0.0.1:11436` | Keeps both inference processes on distinct loopback sockets |
+| `OLLAMA_NO_CLOUD` | `1` for local-only deployments | Makes the local-only privacy boundary explicit |
+| `OLLAMA_FLASH_ATTENTION` | auto | Ollama enables it automatically on supported backends. Force `1` only after a measured compatibility check; quantized K/V cache requires Flash Attention. |
+| `OLLAMA_KV_CACHE_TYPE` | `q8_0` after qualification | Reduces K/V memory; retain `f16` until each important model passes a quality comparison |
+| `OLLAMA_NUM_PARALLEL` | `1` | Avoids multiplying context memory before measuring same-model concurrency |
+| `OLLAMA_MAX_LOADED_MODELS` | `1` per process | Prevents an automatic multi-model limit from overcommitting an unmeasured host |
+| `OLLAMA_MAX_QUEUE` | Workload-specific, bounded value | Limits Ollama's internal queue independently of FreeLlama admission |
+
+Do not set a global `OLLAMA_CONTEXT_LENGTH` merely to maximize the advertised window. Managed
+FreeLlama tasks send the smallest sufficient request-specific `num_ctx`; a larger context and a
+larger `OLLAMA_NUM_PARALLEL` multiply K/V-cache memory. Direct Ollama clients still follow the
+server's context configuration, which defaults to 4096 tokens in current Ollama.
+
+`freellama doctor` returns `local_conservative_config_posture` as a **non-mutating** portable
+starting profile. It names the source of each value (`observed_process` or `configuration_hint`),
+requires `OLLAMA_NO_CLOUD=1` for a local-only deployment, and recommends one loaded model, one
+parallel stream, and a finite internal queue before benchmarking. It also reports
+`host_runtime_signals` with source and permission scope. Treat unavailable GPU-memory, thermal, or
+power signals as unavailable; do not substitute host RAM or a guessed value.
+
+FreeLlama and Ollama have separate queues. FreeLlama acquires a weighted backend permit and waits
+for at most `--max-queue-wait-seconds`. An admitted request can then enter Ollama's internal queue,
+which is bounded by `OLLAMA_MAX_QUEUE`. Raw compatibility traffic bypasses FreeLlama admission and
+enters the primary Ollama queue directly. Set both limits from observed latency and overload
+behavior; changing one does not configure the other.
+
+The following flow shows where each production setting applies:
+
+```mermaid
+flowchart LR
+    C["Client"] --> FQ["FreeLlama admission<br/>weighted budget + wait deadline"]
+    FQ -->|"refused"| E503["503 server busy"]
+    FQ -->|"admitted"| OQ["Ollama queue<br/>OLLAMA_MAX_QUEUE"]
+    RAW["Raw /api/* or /v1/*"] --> OQ
+    OQ --> S["Ollama scheduler<br/>loaded models + parallel streams"]
+    S --> M["Runner memory<br/>num_ctx x K/V cache"]
+```
+
+These values are conservative starting points, not universal constants. NVIDIA, AMD, Apple, and
+CPU-only hosts must pass hardware validation before promotion. See
+[Ollama and FreeLlama optimization](OLLAMA_SYSTEM_OPTIMIZATION.md) for the ownership boundary and
+[Run models on CPU and GPU](CPU_GPU_ROUTING.md) for device-specific controls.
+
+Put the values in the service manager that owns each Ollama process: launchd on macOS, systemd or
+the container definition on Linux, and the Windows service wrapper. Shell exports and
+`launchctl setenv` are useful diagnostics but are login-session state, not a reboot-persistent
+production configuration. After a real service restart, rerun `doctor` and `scripts/check.sh` and
+require the same visible values before admitting work.
+
+## Create the security and state files
+
+Generate a token without placing the secret in shell history or the process list:
+
+```bash
+freellama auth-token --out ~/.local/share/freellama/auth.token
+```
+
+The command creates a new file with mode `0600` on Unix and refuses to overwrite an existing
+token. Start the service with persistent feedback and authentication:
+
+```bash
+freellama serve \
+  --feedback-file ~/.local/share/freellama/feedback.json \
+  --auth-token-file ~/.local/share/freellama/auth.token \
+  --upstream http://127.0.0.1:11434 \
+  --cpu-upstream http://127.0.0.1:11436 \
+  --cpu-model nomic-embed-text:latest
+```
+
+Set `FREELLAMA_AUTH_TOKEN_FILE` to the same file for the CLI, MCP server, and bundled research
+adapters. Authentication covers both `/_freellama/v1/*` and Ollama-compatible passthrough routes.
+Use `--allow-remote` only with an authentication token. The server refuses unauthenticated
+nonloopback listeners.
+
+The feedback snapshot is versioned, bounded by backend and task type, and replaced atomically.
+Only physically verified warm samples are saved. A corrupt or unsupported snapshot makes startup
+fail instead of silently discarding routing evidence. Use `--ephemeral-feedback` only for
+disposable tests.
+
+## Configure MCP
+
+Set these values in the MCP host environment:
+
+- `FREELLAMA_AUTH_TOKEN_FILE`: bearer-token file used by native and adapter HTTP clients.
+- `FREELLAMA_AGENT_TOKEN_CALIBRATION_DIR`: persistent, per-model token-estimate records.
+- `FREELLAMA_MCP_ALLOWED_ROOTS`: exact directories available to `delegate_research`.
+- `FREELLAMA_SERVE_ENDPOINT`: authenticated FreeLlama endpoint.
+
+Persistent token calibration eliminates the repeated “uncalibrated first call” after a model has
+been observed on that host. A genuinely new model/template still starts from the conservative
+configurable estimate because Ollama exposes no stable preflight tokenizer API. Records contain
+only model identifiers, scale, and sample count; they contain no prompts.
+
+Keep adapter context settings distinct from Ollama server context. `FREELLAMA_AGENT_NUM_CTX`
+budgets the adapter conversation; older tool observations are compacted inside that budget while
+the system prompt and original question remain pinned by default. `OLLAMA_CONTEXT_LENGTH` controls
+direct Ollama requests, and managed FreeLlama tasks send their own `num_ctx`.
+
+## Verify lifecycle and placement
+
+`keep_alive:0` no longer sacrifices placement evidence. FreeLlama temporarily holds the runner,
+observes `/api/ps`, records eligible feedback, explicitly unloads it, and verifies that it is no
+longer resident before returning. Inspect both `execution.observation` and
+`execution.lifecycle.status`.
+
+Require these health contracts before admitting traffic:
+
+- `placement_feedback_persistence: versioned_atomic_snapshot_v1`
+- `authentication: optional_bearer_all_routes`
+- `immediate_unload_observation: observe_then_unload`
+- `placement_observation: ollama_api_ps_after_execution`
+- `placement_evidence_gate: configured_or_observed`
+
+Run `skills/freellama/scripts/check.sh` with `FREELLAMA_AUTH_TOKEN_FILE` set. It verifies the
+active versions, visible Ollama settings, persisted-feedback contract, authentication contract,
+backend topology, and binary freshness. Also inspect the service definition for any setting that
+the Ollama process does not expose to FreeLlama.
+
+## Validate every hardware class
+
+Run compile and deterministic test checks from clean checkouts on Windows, Linux, and macOS. Live
+accelerator acceptance is separate because build machines do not necessarily represent production
+GPU drivers or memory topology. Prepare Apple Metal, NVIDIA Linux, AMD Linux, and NVIDIA Windows
+hosts, then run `benchmark/hardware/run_validation.py` directly on each host.
+
+Each environment supplies its exact GPU and CPU model tags and a running FreeLlama endpoint. The
+portable runner in `benchmark/hardware/run_validation.py` executes GPU coding and CPU embedding
+concurrently, verifies physical processor receipts, checks admission receipts, and writes a JSON
+artifact. Add `--vision-model` and `--vision-image` for that host's OCR/vision gate.
+
+Do not mark a hardware row validated until its archived receipt says `verdict: accept`. Missing
+hardware is an explicit release gap, not a result that documentation can waive.
+
+## Promotion checklist
+
+Promote a release only when all conditions pass:
+
+1. `cargo fmt --all --check`, Clippy, Rust tests, typecheck, unit, integration, and live E2E pass.
+2. Release packaging contains every declared target and `SHA256SUMS` verifies.
+3. The active CLI and Ollama server versions match.
+4. Reboot-persistent service definitions carry the explicit Ollama settings and FreeLlama args.
+5. Health reports authentication, atomic feedback persistence, and current placement contracts.
+6. Restart testing proves feedback reloads and a corrupt snapshot fails startup.
+7. Immediate unload reports verified pre-unload placement and verified post-unload absence.
+8. Every claimed hardware row has a real archived acceptance receipt.
+9. The MCP schema remains eight tools and within its context budget.
+
+Rollback by returning the listener to loopback, stopping the new service, and starting the prior
+checksummed binary with the same policy and feedback snapshot. Never downgrade across an unsupported
+feedback schema without preserving the old binary and snapshot together.