@octocodeai/freellama-mcp-server 0.5.0 → 0.6.1

package/docs/MODEL_METADATA.md

@@ -1,106 +0,0 @@
-# Ollama model metadata
-
-The MCP `models` tool gives an agent the inventory and capability metadata it needs before
-offloading work. It reads your Ollama instance and can enrich results with descriptions from
-Ollama's public model library. Supported features and advertised use cases are metadata;
-exact-model benchmarks and policy supply quality evidence. Public descriptions do not change
-routing confidence, and a successful response still needs task-specific verification.
-
-## Requests
-
-| Request | Result | External access |
-|---|---|---|
-| `models {view:"installed"}` | Managed inventory with a `knowledge` object for each model | Local backend data only |
-| `models {view:"installed", includeLibrary:true}` | Inventory enriched with public guidance for verified local tags | Fetches model family and tag pages from ollama.com |
-| `models {view:"detail", model:"gemma3:1b"}` | Ollama details, maximum context, and supported-use explanations | Selected Ollama endpoint only |
-| `models {view:"detail", model:"gemma3:1b", includeLibrary:true}` | Details plus public guidance after a tag and digest check | Selected Ollama endpoint and ollama.com |
-| `models {view:"library", query:"coding"}` | Public family search with descriptions and advertised feature chips | ollama.com and optional local inventory/profile reads |
-| `models {view:"library", model:"gemma3"}` | All public tags with size, digest prefix, context, modalities, and cloud status | ollama.com and optional local inventory/profile reads |
-
-These are MCP tool requests. The CLI's `models` command reports the core catalog and does not expose these enrichment flags. For CLI setup and routing, see the [CLI reference](CLI.md).
-
-| Option | Valid views | Default | Meaning |
-|---|---|---|---|
-| `includeLibrary` | `installed`, `detail` | `false` | Requests public metadata; the default makes no website calls |
-| `includeReadme` | Library step 2, or `detail` with `includeLibrary:true` | `false` | Returns README text, capped at 65,536 characters, with its format and truncation flag |
-| `includeVerbose` | `detail` | `false` | Returns Ollama's license and Modelfile fields independently of README enrichment |
-| `limit`, `cursor` | `raw`, library tag listings; `limit` also applies to library search | Tag/raw page size 20; search limit 10 | Pages large lists; the tag cursor rejects a changed tag list |
-| `ollamaEndpoint` | Direct Ollama views and library inventory reads | Configured Ollama endpoint | Chooses the Ollama instance; enriched managed inventory uses each model's assigned backend |
-
-An incompatible option returns an error before any lookup. Installed enrichment omits full READMEs to keep inventory responses bounded; request one model's detail to read its README.
-
-## Supported features
-
-`knowledge.supportedFeatures` preserves the feature strings available from the selected source. `supportedFeaturesSource` is `ollama_api` for direct Ollama metadata and `freellama_catalog` for the managed catalog. The catalog contains normalized routing capabilities; unknown future strings remain visible in `detail`, `raw`, or enriched installed inventory when Ollama's tag response supplies them. Unknown strings do not create new routing requirements.
-
-`knowledge.supportedUseCases` explains each recognized feature:
-
-| Feature | Supported workflow | What to evaluate separately |
-|---|---|---|
-| `completion` | Text generation | Coding, summarization, reasoning, and instruction-following accuracy |
-| `tools` | Tool calls | Argument correctness, tool selection, and agent-loop reliability |
-| `vision` | Image input | OCR and visual understanding accuracy |
-| `audio` | Audio workflows advertised by the model | Supported audio task and accuracy |
-| `embedding` | Retrieval and similarity vectors | Retrieval quality on your documents |
-| `thinking` | A thinking mode | Reasoning accuracy, latency, and token cost |
-| `insert` | Suffix-based text insertion | Completion accuracy in the target code or text |
-| `image` | Image generation | Output quality and supported controls |
-
-Each explanation contains its `capability`, `use`, `reason`, and `basis:"supported_feature"`. Ollama runtime/version support still applies. The inventory's existing benchmark and policy fields remain separate from `knowledge.quality:"metadata_only"`.
-
-## Public guidance and provenance
-
-For a verified installed model, `knowledge.library` contains the source page, full tag-page URL, `fetchedAt`, `expiresAt`, `cached`, and exact public tag metadata. It also contains:
-
-| Field | Meaning |
-|---|---|
-| `description` | The public family description |
-| `advertisedFamilyCapabilities` | Family feature chips; these do not extend the local tag's capabilities |
-| `advertisedUseCases` | Recognized coding, tool-calling, image-understanding/OCR, reasoning, embedding, summarization, question-answering, translation, and long-context claims |
-| `advertisedUseCases[].evidence` | A bounded source excerpt, source URL, and fetch timestamp |
-| `usageNotes` | Bounded excerpts about requirements, restrictions, context, memory, languages, or query prefixes |
-| `readmeAvailable` | Whether a README was extracted |
-| `readme` | Requested text with `format` (`markdown` or `text`) and `truncated` |
-
-Advertised uses have `basis:"advertised"`, `scope:"family"`, and `quality:"unmeasured"`. Extraction recognizes positive prose mentions and skips fenced code and explicit negative statements. It is a discovery aid, not a complete classification: an omitted use means extraction found no recognized claim. Usage notes also remain family-scoped and can describe other variants. Read the source for the full instructions. These use labels are not the `run_task` task enum; choose a supported routing profile with the capabilities and context your workload requires.
-
-Local capabilities filter advertised uses. For example, `gemma3:1b` accepts text, while larger Gemma 3 variants accept images. A family-level vision chip cannot make the 1B tag a vision candidate. Likewise, an embedding-only tag does not acquire text-generation workflows from its family description.
-
-Installed enrichment requires the exact public tag and a matching digest prefix of at least 12 hexadecimal characters against the local 64-character digest. Custom names are never mapped by architecture or guessed base model. Community names use their exact Ollama namespace. Without a verified match, public guidance is withheld. A local modification can therefore prevent enrichment even when its name resembles a public model.
-
-Library step 2 describes public models before installation, so its top-level `library` guidance remains family-scoped without claiming a local digest match. Each `tags[]` entry carries its own modalities, context, digest prefix, and `cloud` flag. Search returns `cloudAvailable` for the hosted-access badge and `cloudOnly:null`: a family badge alone does not establish that local downloads are absent.
-
-## Lookup states
-
-| `library.status` | Meaning | Next action |
-|---|---|---|
-| `not_requested` | Website enrichment was not requested | Set `includeLibrary:true` if external lookup is useful |
-| `matched` | Installed tag and public digest prefix match | Read the sourced guidance, then evaluate the exact tag |
-| `available` | Public family and tag pages were parsed in library step 2 | Inspect the exact tag before recommending a download |
-| `unverified` | Digest or current backend inventory could not be verified | Recheck local inventory; retain local feature information |
-| `digest_mismatch` | The public tag differs from the installed digest | Treat public guidance as unavailable for this installed model |
-| `tag_not_found` | The parsed public tag list has no exact matching tag | Check the source and custom-model provenance |
-| `unmatched` | The name cannot resolve to an exact public Ollama name | Keep use-case suitability unknown until provenance is verified |
-| `not_found` | A requested public page returned HTTP 404 | Check the source URL or custom name |
-| `unavailable` | Transport, timeout, HTTP refusal, or page-size limit prevented lookup | Retry later; local results remain available |
-| `parse_error` | HTML content or tag rows were not recognized | Inspect the source; do not conclude the model is absent |
-| `remote_model`, `remote_tag` | Remote Ollama fields or a hosted tag identify cloud execution | Select a local download tag for local execution |
-
-`localModel:false` flags detected remote entries. Known remote entries skip public enrichment, and cloud tags never become local trial recommendations. This metadata is not processor-placement evidence; inspect the execution observation for CPU/GPU placement.
-
-## Cache and request bounds
-
-Each MCP process caches at most 128 families. Successful lookups expire after one hour, failures after one minute; concurrent requests for a family share one lookup. Expired content is not served as current metadata. Restarting the MCP process clears this in-memory cache.
-
-Installed enrichment runs at most four family lookups concurrently. Each lookup requests the family page and complete `/tags` page in parallel. Requests use GET, refuse redirects, cap each page at 2 MiB, and time out after the smaller of 10 seconds and `FREELLAMA_MCP_FETCH_TIMEOUT_SECONDS`. These requests retrieve metadata without loading or executing a model. They disclose the requested public family name to ollama.com.
-
-The local inventory is not cached by the enrichment layer. A changed backend inventory or failed lookup leaves the managed model and its existing evidence intact. A size-based library recommendation is only a download trial candidate: download size does not establish runtime memory fit, CPU/GPU placement, or task quality. Model discovery never authorizes a pull.
-
-## Sources and related documentation
-
-- [Ollama model details API](https://docs.ollama.com/api-reference/show-model-details) provides runtime capability and architecture metadata.
-- [Ollama model-list API](https://docs.ollama.com/api/tags) provides installed tags and digests.
-- [Ollama Gemma 3 page](https://ollama.com/library/gemma3) illustrates variant-specific modalities.
-- [MCP tool reference](../packages/mcp/README.md#tools) documents the complete tool surface.
-- [Choose local models](MODEL_SELECTION.md) explains quality qualification and measured results.
-- [Test FreeLlama](TESTING.md) lists the parser, enrichment, and protocol checks.

package/docs/OLLAMA_SIDECAR.md

@@ -1,78 +0,0 @@
-# Ollama sidecar boundary
-
-FreeLlama exposes task-oriented controls to agents and communicates with Ollama over HTTP.
-It defaults to a loopback sidecar: FreeLlama manages delegation policy and resources around each
-managed request, while Ollama loads the runner and performs inference.
-
-```mermaid
-flowchart LR
-    C["Ollama-compatible client"] --> F["FreeLlama loopback sidecar"]
-    F -->|"raw /api and /v1"| G["Primary Ollama"]
-    F -->|"managed primary task"| G
-    F -->|"managed assigned task"| U["Optional CPU Ollama"]
-    G --> E["Metal, MLX, CUDA, ROCm, Vulkan, or CPU runner"]
-    U --> E2["CPU runner"]
-```
-
-This boundary avoids an Ollama fork and preserves its documented HTTP contract. It also makes a
-second, process-isolated CPU backend possible without changing arbitrary client payloads.
-
-## Divide ownership
-
-| Problem | Owner |
-|---|---|
-| Kernels, prompt processing, and token generation | Ollama and its runner dependencies |
-| Runner loading, eviction, and scheduling within one server | Ollama |
-| Prompt templates and model defaults | Ollama model metadata or a `Modelfile` |
-| Capability and memory filtering | FreeLlama managed routes |
-| Evidence policy, confidence, and route receipts | FreeLlama managed routes |
-| Session affinity, per-backend bounded admission, and warm runtime feedback | FreeLlama managed routes |
-| Forecast memory reservations and requests to evict idle unpinned runners | FreeLlama managed routes |
-| Exact model assignment to an optional CPU process | FreeLlama configuration |
-| Guarded CPU/GPU preference within assigned eligible models | FreeLlama managed routes |
-| Authentication and remote exposure | FreeLlama bearer middleware; external ingress owns TLS and tenant authorization |
-
-FreeLlama does not make an individual model decode faster. It can improve workload completion by
-avoiding unsuitable routes, reducing avoidable transitions, refusing unsupported work early, and
-overlapping independent CPU and GPU model requests when the machine benefits from that layout.
-
-## Preserve compatibility
-
-`packages/rust-core/src/proxy.rs` forwards paths, queries, request bodies, and streaming response
-bodies. It removes hop-by-hop headers and adds `x-freellama-proxy: 1` to responses. Raw `/api/*`
-and `/v1/*` requests always target the primary upstream and are not rewritten for CPU placement.
-
-```mermaid
-flowchart TD
-    R["Incoming request"] --> K{"Managed /_freellama/v1 route?"}
-    K -->|"No"| P["Stream unchanged to primary Ollama"]
-    K -->|"Yes"| V["Validate, route, assign backend, and admit"]
-    V --> X["Build bounded Ollama request"]
-    X --> O["Execute on assigned upstream"]
-```
-
-The distinction is intentional: existing Ollama clients retain predictable compatibility, while
-clients that opt into the managed API receive routing and policy behavior.
-
-## Know the remaining limits
-
-In `serve`, mutating passthrough traffic shares the primary transition lock and receives host-pressure
-gating, while managed tasks also reserve forecast memory. Direct Ollama callers bypass these
-controls. FreeLlama cannot preempt inference or constrain other processes. A configured but unreachable CPU backend makes managed catalog discovery fail closed;
-raw primary passthrough remains usable.
-
-Runtime feedback is task-specific, token-normalized, and warm-only. It can steer
-unpinned `fastest` or `balanced` work after three samples on each backend and a greater-than-10%
-advantage; it cannot change CPU eligibility or steer quality routing. The CLI persists a bounded,
-versioned snapshot atomically by default, so verified evidence survives restart without changing
-Ollama residency.
-
-Read [Architecture](ARCHITECTURE.md) for all managed flows,
-[CPU and GPU model routing](CPU_GPU_ROUTING.md) for the two-process procedure, and
-[Ollama and FreeLlama optimization](dev/OLLAMA_SYSTEM_OPTIMIZATION.md) for the full tuning boundary.
-
-## Sources
-
-- [Ollama API introduction](https://docs.ollama.com/api/introduction)
-- [Ollama streaming behavior](https://docs.ollama.com/api/streaming)
-- [Ollama usage metrics](https://docs.ollama.com/api/usage)

package/docs/PRODUCT_POSITIONING.md

@@ -1,121 +0,0 @@
-# Product positioning
-
-FreeLlama gives AI agents a managed way to offload tasks to local models through Ollama.
-Agents submit bounded work through MCP; FreeLlama handles model selection, queues, memory checks,
-residency coordination, and routing to configured CPU/GPU backends. It returns results with
-execution evidence so the calling agent can verify them.
-
-> Agents delegate tasks. FreeLlama manages local resources. Ollama runs inference.
-
-The CLI and control API expose the same management layer to operators and applications.
-See [architecture](ARCHITECTURE.md) for the ownership contract and the
-[README](../README.md) for installation and first use.
-
-## Audience and purpose
-
-The primary audience is developers operating coding or research agents with Ollama. Typical work
-includes grounded repository lookup, embeddings, image/OCR tasks, supplied-content transforms,
-and helper-model requests. Local-AI operators can use the CLI for the same routing and resource
-controls without an MCP host.
-
-The purpose is to make suitable local delegation bounded and inspectable. Eligibility checks,
-policy, and benchmark evidence help an agent choose a model; they do not prove that an individual
-answer is correct. The calling agent retains judgment and final verification.
-
-## How the layers fit
-
-```mermaid
-flowchart LR
-    A["Calling agent: choose task and constraints"] --> F["FreeLlama: select model and manage resources"]
-    F -->|"managed primary task"| G["Primary Ollama"]
-    F -->|"assigned CPU helper"| C["Optional CPU Ollama"]
-    F -->|"capacity unavailable"| Q["Bounded queue or refusal"]
-    G --> R["Result and execution evidence"]
-    C --> R
-    R --> V["Calling agent: verify and continue"]
-```
-
-| Layer | Owns |
-|---|---|
-| Calling agent | What to offload, prompts, task dependencies, independent concurrency, and answer verification |
-| FreeLlama | Installed-model eligibility, policy, managed admission, memory forecasts, residency coordination, bounded history, and execution evidence |
-| Operator | Ollama installation, endpoints, exact CPU model assignments, lifecycle approval, and runtime configuration |
-| Ollama and the host | Model storage, runner loading, inference, and physical CPU/GPU scheduling |
-
-Start with one Ollama process. A second process is optional for explicitly assigned CPU helpers.
-An agent's placement preference chooses only among eligible configured models. Inspect observed
-placement before treating CPU/GPU assignment as physical evidence.
-
-## What agents receive
-
-| Need | Surface | Result |
-|---|---|---|
-| Inspect the machine and models | `doctor`, `models` | Host diagnostics, installed capabilities, and resident-runner observations |
-| Preview suitability | `run_task {preview:true}` | A decision and reasons without running inference |
-| Execute supplied content | `run_task` | Model output with routing, admission, timing, and placement evidence |
-| Look up repository facts | `delegate_research` | A bounded read-only answer, citations, and `accept`, `verify`, or `escalate` verdict |
-| Dispatch independent work | `run_task_batch` | Bounded dispatch and separate results or errors |
-| Manage longer work | `task_jobs` | Process-local status, result retrieval, cancellation, and removal |
-| Reuse conversation history | `scope`, `session`, `warm_model` | Separate controls for message history, model affinity, and residency |
-
-Preview accepts routing fields only. Submit prompts, messages, images, embedding input, tools,
-and runtime controls in a separate execution call. A preview does not reserve capacity.
-For requests and result fields, use the [MCP reference](../packages/mcp/README.md).
-
-## Resource management
-
-FreeLlama manages requests before they reach the inference runtime. Its resource controls include:
-
-- Per-backend weighted admission, finite queues, priorities, wait budgets, and deadlines.
-- Host-pressure checks and forecast memory reservations for managed work.
-- Model-transition coordination and requests to unload idle, unpinned models when a cold load needs room.
-- Explicit routing to the primary or operator-assigned CPU backend.
-- Bounded warm execution feedback for eligible speed-oriented automatic routes.
-- Receipts and status views that expose queues, resource holds, residency, timing, and placement.
-
-The operator configures topology and limits. FreeLlama does not preempt inference or control other
-processes that call Ollama directly. It coordinates model use through Ollama rather than allocating
-physical GPU cores or implementing inference kernels.
-
-Read [CPU/GPU routing](CPU_GPU_ROUTING.md) for placement and
-[monitoring](MONITORING.md) for effective settings and runtime observations.
-
-## Context and cost
-
-Delegated research keeps source files and intermediate local-tool observations out of the calling
-agent's prompt. The local worker still consumes input/output tokens and hardware time. Embedding
-vectors stay out of MCP results by default unless the caller requests them.
-
-This can reduce the calling agent's context burden. It does not imply zero-cost computation or a
-universal savings rate. The [token economics report](ECONOMICS.md) separates historical measurements,
-client-envelope sizes, and session estimates.
-
-## Describe the product consistently
-
-| Context | Wording |
-|---|---|
-| Category | Local-model delegation and resource management layer |
-| One line | FreeLlama helps AI agents offload tasks to local models while managing model selection and resources. |
-| Mechanism | Inspect, qualify, preview, admit, execute, and return evidence. |
-| Runtime relationship | FreeLlama manages requests around Ollama; Ollama runs inference. |
-| Verification | The calling agent verifies answers against the task and returned evidence. |
-
-Describe model eligibility, verification controls, and measured workload performance. Describe
-CPU/GPU overlap as a supported workload arrangement that needs measurement on
-the target machine. Local inference consumes hardware, memory, power, and time.
-
-The name refers to reducing reliance on metered inference. Models are not restricted to the Meta
-Llama family: FreeLlama works with eligible models exposed by Ollama.
-
-## Evidence and boundaries
-
-The [findings report](dev/FINDINGS_AND_POSITIONING.md) records dated assessments and comparisons.
-[Benchmarks](../benchmark/README.md) separate model correctness, adapter behavior, and hardware
-validation. Historical measurements describe their declared model, machine, and workload.
-
-Managed tasks are non-streaming; raw Ollama passthrough retains upstream streaming behavior.
-Scopes and deferred jobs are process-local. A scope is history, a session is affinity, and a warm
-model is residency; none is a tenant credential or an engine-cache snapshot.
-
-FreeLlama targets one operator or a trusted team. Use the [production runbook](PRODUCTION.md) for
-authentication, remote access, persistence, and recovery.

package/docs/dev/ADR_RESOURCE_AWARE_BACKEND_ROUTING.md

@@ -1,55 +0,0 @@
-# ADR: Resource-aware backend routing
-
-Status: Accepted and implemented
-
-## Context
-
-FreeLlama can isolate explicitly assigned helper models in a second Ollama process. Live trials
-proved that CPU and GPU requests can overlap, but exact operator assignment alone was too rigid for
-agents, and one global admission pool can let GPU work starve a CPU helper. Unconstrained
-automatic migration is unsafe because capability, quality, session, and memory constraints
-must continue to win.
-
-## Decision
-
-Keep backend eligibility operator-owned and add three bounded controls:
-
-1. Callers may send `auto`, `prefer_cpu`, or `prefer_gpu`; this is a fallback-capable hint.
-2. GPU and CPU backends own independent weighted admission pools and transition locks.
-3. `auto` can use task-specific normalized warm latency after three samples exist on both backends
-   and one backend has more than a 10% advantage.
-
-Explicit models and session affinity override the hint. Quality routing never follows latency.
-Generation normalizes by output tokens; embeddings normalize by input tokens. Cold-load duration
-does not enter feedback. Raw Ollama passthrough remains byte-transparent to the primary upstream.
-Verified feedback is stored in a bounded, versioned snapshot and replaced atomically. A corrupt or
-unsupported snapshot fails startup instead of silently changing learned routing behavior.
-
-The contract is hardware-neutral: the operator supplies loopback endpoints and exact CPU-eligible
-tags, while FreeLlama discovers host capacity and observes Ollama placement. The measured 48 GB Mac
-result is validation evidence, not a routing constant. Device-visibility variables remain
-Ollama-process configuration because NVIDIA, ROCm, Vulkan, Metal, and CPU-only hosts differ.
-
-## Alternatives
-
-| Alternative | Decision | Reason |
-|---|---|---|
-| Exact assignment only | Rejected | Safe but too rigid for agents and blind to runtime capacity |
-| Let agents supply an upstream or `num_gpu` | Rejected | Bypasses operator eligibility and compatibility boundaries |
-| Rewrite every raw Ollama request | Rejected | Breaks passthrough semantics and surprises existing clients |
-| Persist every raw runtime event | Rejected | Unbounded traces add privacy and storage risk; persist only aggregate prompt-free feedback |
-| Live memory/thermal controller | Deferred | No stable sensor and held-out workload yet justify it |
-
-## Verification contract
-
-- Preview returns placement, upstream, preference satisfaction, reason, and admission capacity.
-- Two samples per backend do not change `auto`; the third makes the comparison decision-ready.
-- Explicit model and session pins do not move.
-- A one-unit GPU pool and one-unit CPU pool admit one task on each concurrently.
-- CPU-managed requests contain `options.num_gpu: 0`; raw `/api/*` bodies are unchanged.
-- Health exposes both pools and per-task feedback counts.
-- Restart reloads the same bounded feedback counts; an unsupported schema refuses startup.
-
-Rollback is configuration-only for callers: omit the preference or use `auto`. Operators can remove
-the CPU backend to restore a single-primary layout. Reverting independent pools requires a code
-rollback and is not recommended because it reintroduces cross-backend starvation.