@octocodeai/freellama-mcp-server 0.1.0 → 0.5.0

package/docs/{OLLAMA_SYSTEM_OPTIMIZATION.md → dev/OLLAMA_SYSTEM_OPTIMIZATION.md}

@@ -1,8 +1,8 @@
 # Ollama and FreeLlama optimization boundary
 
-Ollama already owns inference optimization. FreeLlama can improve completed
-work per hour by choosing an eligible model and avoiding unnecessary model
-changes, but it does not make an individual model decode faster.
+FreeLlama manages model selection and resources for tasks offloaded by agents. Ollama owns
+inference optimization. FreeLlama can improve completed work per hour by avoiding unsuitable
+routes and unnecessary model transitions; measure that effect for the target workload.
 
 This page explains the active request flow, the settings Ollama owns, the
 current machine audit, and the optimization work that remains.
@@ -19,8 +19,9 @@ control plane for evidence-qualified routing, bounded request profiles, session
 affinity, and policy enforcement.
 
 Do not advertise FreeLlama as an accelerator or an Ollama inference plugin. It does not tune
-kernels, split one model's layers across CPU and GPU, manage macOS memory pressure, or improve that
-model's tokens per second. It can assign different models to isolated CPU and GPU-capable Ollama
+kernels or split one model's layers across CPU and GPU. It responds to observed host pressure by
+holding managed work; the operating system manages physical memory. FreeLlama can assign different
+models to isolated CPU and GPU-capable Ollama
 processes, allowing independent workloads to overlap.
 
 ## Request flow
@@ -86,10 +87,10 @@ request-specific `num_ctx` from the selected task profile. The coding adapters f
 conversation inside `FREELLAMA_AGENT_NUM_CTX`: by default, they preserve the system prompt and
 original question, compact the oldest observations first, retain recent evidence, and fail before
 the Ollama call if pinned content cannot fit. Raising the Ollama server's global context does not
-disable adapter compaction or increase its configured budget. See [Agent adapters](../AGENTS.md)
+disable adapter compaction or increase its configured budget. See [adapter contracts](../../benchmark/local/docs/07-adapter-contracts.md)
 for the complete pagination and compaction contract.
 
-## Current Mac audit
+## Mac audit (2026-09-01)
 
 This audit captures the following snapshot from September 1, 2026. Treat it as machine
 evidence, not a portable default.
@@ -164,15 +165,15 @@ than an inference-speed claim.
 
 FreeLlama does not provide:
 
-- a live minimum-free-VRAM guard. FreeLlama does calculate a metadata-backed F16 K/V lower bound
-  and refuses a known impossible CPU/unified-memory floor, but Ollama remains the authority for
-  current free memory, runner graph allocation, and final loading;
-- transition coordination for native passthrough requests or processes that
-  bypass the managed `tasks` endpoint;
-- live thermal, power-mode, or swap-aware routing;
-- online token-rate, error-rate, or cold-load-duration scoring;
+- exact runner-allocation prediction. Managed admission uses observed available RAM, free VRAM
+  when available, and a metadata-backed F16 K/V estimate; unknown layouts remain unknown. Ollama
+  remains authoritative for runner allocation and final loading;
+- coordination for processes that bypass FreeLlama. Mutating passthrough requests in `serve`
+  share the primary transition lock;
+- direct control over thermal state, power mode, or swapping. Observed pressure can hold admission;
+- model-quality scoring inferred from operational latency or errors;
 - per-engine tuning for GGUF compared with MLX;
-- dynamic `keep_alive` or concurrency limits by workload;
+- unbounded automatic residency or concurrency. Runtime limits remain operator-owned;
 - an atomic natural-language route-and-run operation;
 - held-out quality policies for completion, coding, tools, browser, vision, or
   long-context tasks in the example policy;
@@ -200,7 +201,7 @@ Use this table before changing Ollama or FreeLlama:
 | Leave Ollama's queue without a deployment limit | Do not add | Set a measured `OLLAMA_MAX_QUEUE`; it remains separate from FreeLlama's admission deadline |
 | Pin multiple 18–21 GB models | Do not add by default | The observed 36 GiB Metal budget cannot safely establish that two heavy models plus contexts fit |
 | Force Flash Attention | Benchmark per engine and model | The inspected GGUF path already enabled it automatically; MLX has a different runner |
-| Use `q8_0` K/V cache | **8/10; qualify, then prefer for parallel/long-context work** | Ollama documents roughly half the KV memory of `f16` with very small precision loss; it is the practical companion to higher `OLLAMA_NUM_PARALLEL`, but it remains a process-wide quality tradeoff |
+| Use `q8_0` K/V cache | Qualify quality and memory effects per workload | KV quantization changes memory use and precision for the whole process; compare representative task quality and resident memory before deployment |
 | Use `q4_0` K/V cache | Reject as a default | It saves more memory with a larger possible quality loss |
 | Enable High Power Mode | Sustained-load experiment only | Apple documents higher sustained performance. Local token-rate and energy effects are not measured |
 | Keep the small intent model resident | Keep bounded | It reduces routing latency but consumes memory and can affect a large-model transition |
@@ -222,10 +223,10 @@ memory, swap, and failure counts. A higher token rate is not an improvement if
 quality falls, requests fail, or model transitions dominate the workload.
 
 Use `cargo run -- bench-all` (`packages/rust-core/src/model_bench.rs`) for model selection evidence.
-Use [CPU and GPU model routing](CPU_GPU_ROUTING.md) for the separate-process benchmark and
+Use [CPU and GPU model routing](../CPU_GPU_ROUTING.md) for the separate-process benchmark and
 verification procedure.
 
-- Local-model router RFC — lived in the gitignored `.octocode/` workspace and is no longer in the tree; the shipped design it argued for is [ARCHITECTURE.md](ARCHITECTURE.md)
+- Local-model router RFC — lived in the gitignored `.octocode/` workspace and is no longer in the tree; the shipped design it argued for is [ARCHITECTURE.md](../ARCHITECTURE.md)
 
 ## Recommended next work
 

package/native/index.d.ts

@@ -3,6 +3,9 @@
 // exactly — keep the two in sync if you change one. `npm run build` no longer overwrites this
 // file (it previously did, silently emptying it — see package.json's build script comment).
 
+/** Crate version this native addon was compiled from. */
+export function version(): string;
+
 /**
  * Runs `freellama doctor` against Ollama directly — no running `freellama serve` required.
  * Cross-checks the Ollama CLI and server versions and confirms the endpoint is reachable.
@@ -20,9 +23,20 @@ export function machine(endpoint?: string | undefined | null): Promise<string>;
 /** Current managed-platform health, including admission and session bounds. */
 export function health(endpoint?: string | undefined | null): Promise<string>;
 
+/**
+ * Live platform view: admission, queues, adaptive limits and circuit breakers per backend,
+ * loaded models, host memory, Ollama's effective settings, and today's usage.
+ */
+export function status(endpoint?: string | undefined | null): Promise<string>;
+
+/** Usage totals per day and per model over the last `days` days (default 7). */
+export function usage(endpoint?: string | undefined | null, days?: number | undefined | null): Promise<string>;
+
 /** Create a bounded, idle-expiring model-affinity handle; it stores no prompt history or KV. */
 export function createSession(endpoint?: string | undefined | null): Promise<string>;
 
+export function killSession(endpoint: string | undefined | null, sessionId: string): Promise<string>;
+
 /** Release a model-affinity handle after related work has completed. */
 export function deleteSession(
   endpoint: string | undefined | null,
@@ -145,6 +159,15 @@ export function runTaskBatchRequest(
   request: unknown,
 ): Promise<string>;
 
+/** Lists process-local job metadata without prompts or results. */
+export function listTaskJobs(endpoint?: string | undefined | null): Promise<string>;
+/** Reads one deferred job and its retained result. */
+export function getTaskJob(endpoint: string | undefined | null, jobId: string): Promise<string>;
+/** Cancels one deferred job and waits for its local permits to be released. */
+export function cancelTaskJob(endpoint: string | undefined | null, jobId: string): Promise<string>;
+/** Stop one deferred task, release local permits, then discard its retained record. */
+export function removeTaskJob(endpoint: string | undefined | null, jobId: string): Promise<string>;
+
 /**
  * Converts a free-text natural-language intent into a route.
  * @param endpoint FreeLlama serve endpoint, defaults to http://127.0.0.1:11435
@@ -156,3 +179,11 @@ export function naturalRoute(
   text: string,
   sessionId?: string | undefined | null,
 ): Promise<string>;
+
+/** Opt-in bounded process-local history. Metadata by default. */
+export function createScope(endpoint: string | undefined | null, request: unknown): Promise<string>;
+export function getScope(endpoint: string | undefined | null, scopeId: string, includeMessages?: boolean | null): Promise<string>;
+export function forkScope(endpoint: string | undefined | null, scopeId: string, request: unknown): Promise<string>;
+export function deleteScope(endpoint: string | undefined | null, scopeId: string): Promise<void>;
+/** Empty-payload preload through managed admission and deadlines. */
+export function warmModelRequest(endpoint: string | undefined | null, request: unknown): Promise<string>;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@octocodeai/freellama-mcp-server",
-  "version": "0.1.0",
+  "version": "0.5.0",
   "description": "MCP server exposing FreeLlama core as tools, via native NAPI bindings into packages/rust-core.",
   "repository": {
     "type": "git",
@@ -28,7 +28,10 @@
     "native/package.json",
     "adapters/*.py",
     "docs",
-    "README.md"
+    "README.md",
+    "LICENSE-APACHE",
+    "LICENSE-MIT",
+    "assets/logo.jpg"
   ],
   "scripts": {
     "build": "node scripts/build.mjs",
@@ -46,14 +49,14 @@
     "zod": "^3.24.1"
   },
   "optionalDependencies": {
-    "@octocodeai/freellama-native-darwin-arm64": "0.1.0",
-    "@octocodeai/freellama-native-darwin-x64": "0.1.0",
-    "@octocodeai/freellama-native-linux-arm64-gnu": "0.1.0",
-    "@octocodeai/freellama-native-linux-arm64-musl": "0.1.0",
-    "@octocodeai/freellama-native-linux-x64-gnu": "0.1.0",
-    "@octocodeai/freellama-native-linux-x64-musl": "0.1.0",
-    "@octocodeai/freellama-native-win32-arm64-msvc": "0.1.0",
-    "@octocodeai/freellama-native-win32-x64-msvc": "0.1.0"
+    "@octocodeai/freellama-native-darwin-arm64": "0.5.0",
+    "@octocodeai/freellama-native-darwin-x64": "0.5.0",
+    "@octocodeai/freellama-native-linux-arm64-gnu": "0.5.0",
+    "@octocodeai/freellama-native-linux-arm64-musl": "0.5.0",
+    "@octocodeai/freellama-native-linux-x64-gnu": "0.5.0",
+    "@octocodeai/freellama-native-linux-x64-musl": "0.5.0",
+    "@octocodeai/freellama-native-win32-arm64-msvc": "0.5.0",
+    "@octocodeai/freellama-native-win32-x64-msvc": "0.5.0"
   },
   "license": "Apache-2.0 OR MIT",
   "publishConfig": {