@octocodeai/freellama-mcp-server 0.1.0 → 0.5.0

package/docs/SCOPES_AND_WARMING.md

@@ -0,0 +1,226 @@
+# Scope history and model warming
+
+Use scopes when delegated chat tasks need retained message history, and warming when an installed
+model needs preparation before related work. FreeLlama manages these controls through the same
+admission and memory checks as task execution. The calling agent owns conversation instructions,
+task dependencies, and verification.
+
+## Ownership
+
+| State | Owner | Lifetime and meaning |
+|---|---|---|
+| Session affinity | FreeLlama `session` | Bounded, idle-expiring model preference; stores no messages or KV. |
+| Scope history | FreeLlama `scope` | Bounded process-local messages and routing defaults, protected by a revision. |
+| Runner residency | Ollama | Controlled by `keep_alive`, loading, eviction, and available resources. |
+| Prefix cache and KV | Ollama and its inference backend | Opportunistic reuse of compatible requests; no scope-owned KV save/restore contract. |
+
+Reusing a scope replays its stored messages. Switching scope IDs does not transfer KV tensors.
+Keeping an identical message prefix can preserve cache reuse when the backend supports it; changing
+the model, runner profile, or prefix can invalidate that reuse. A resident model does not imply that
+the requested history is cached. Cache counts remain unknown when Ollama does not report them.
+
+## Scope API
+
+All HTTP paths below are relative to `/_freellama/v1`. Authentication matches the other control
+routes. Bodies reject unknown fields.
+
+| Operation | HTTP | MCP | CLI |
+|---|---|---|---|
+| Create | `POST /scopes` | `scope {action:"create"}` | `scope create` |
+| Inspect metadata | `GET /scopes/:id` | `scope {action:"get",scopeId}` | `scope get --scope-id SCOPE_ID` |
+| Inspect history | `GET /scopes/:id?include_messages=true` | `scope {action:"get",scopeId,includeMessages:true}` | `scope get --scope-id SCOPE_ID --include-messages` |
+| Fork a snapshot | `POST /scopes/:id/fork` | `scope {action:"fork",scopeId,revision}` | `scope fork --scope-id SCOPE_ID --revision REVISION` |
+| Delete | `DELETE /scopes/:id` | `scope {action:"delete",scopeId}` | `scope delete --scope-id SCOPE_ID` |
+
+Replace `SCOPE_ID` with the returned scope identifier and `REVISION` with its current revision.
+Deletion returns HTTP `204`. Create and fork return HTTP `201` with metadata; history requires an
+explicit read. CLI create and fork accept a JSON object through `--json`.
+
+Create accepts the following optional fields:
+
+| HTTP field | MCP field | Meaning |
+|---|---|---|
+| `messages` | `messages` | Initial Ollama messages, preserving fields, content, and order. |
+| `route_defaults` | `routeDefaults` | Task-routing defaults for later scoped execution. |
+| `limits` | `limits` | Per-scope bounds within the server's configured limits. |
+
+Messages require a supported role and can omit content, including for assistant tool calls.
+When present, content must be a string.
+
+Routing defaults accept `task`, `objective`, `model`, `required_capabilities`, `context_tokens`,
+`execution_preference`, `min_placement_evidence`, and `min_confidence`. MCP uses the corresponding
+camelCase field names. Defaults do not grant admission or qualify a model. Explicit task fields
+take precedence, and execution checks current eligibility and resources.
+
+The metadata receipt contains `scope_id`, `revision`, `route_defaults`, `limits`, `message_count`,
+`bytes`, `estimated_tokens`, `expires_at`, and `storage:"process_local"`. Full history can include images, thinking, and tool calls; request it only when needed. Counts and estimates
+describe the stored transcript; they are not an exact tokenizer result or runner-memory estimate.
+Token estimates derive from the full serialized message bytes and count both media and extra fields.
+This storage-budget estimate can refuse media-heavy history before inference; it does not measure
+image token cost or determine the model context required to execute the history.
+
+Fork requires `revision` and accepts optional replacement routing defaults and limit overrides. It creates a
+separate history snapshot with revision `0`; later writes to either scope do not change the other.
+Fork independent branches before parallel execution instead of submitting concurrent writes to one
+scope.
+
+## Scoped task execution
+
+HTTP `POST /tasks` accepts `scope_id` and `scope_revision` together. MCP `run_task` uses `scopeId`
+and `scopeRevision`; CLI `task` uses `--scope-id` and `--scope-revision`. Send only the new caller
+messages or prompt. FreeLlama prepends the stored transcript and appends the successful assistant
+response to that scope. Embedding tasks and embedding inputs cannot use scopes. MCP previews
+reject scope references; use a separate execution call.
+
+The expected revision prevents a stale caller from overwriting another result. A scope permits one
+active append; a competing write fails before inference. A successful task returns a `scope`
+metadata receipt with the next revision. Failed or cancelled tasks do not append history. A scope
+deleted or expired during execution cannot receive the result. History limits also apply to the
+completed assistant message: a response that exceeds them returns an append error with the prior
+revision intact even though inference has run. Inspect the failure before retrying.
+
+Deleting a session releases its affinity without deleting scope history or invalidating an
+already-running scope append. Killing a session cancels its associated work; cancelled work does
+not append history.
+
+Limits reject an oversized transcript instead of silently deleting earlier instructions. If a task
+cannot fit, narrow its input, create a compact replacement scope, or fork a suitable snapshot. The
+caller owns any semantic summary and its verification. Stored history also contributes to managed
+context sizing; the selected model's context boundary still applies. Scoped image or audio input
+requires explicit `context_tokens` (`contextTokens` through MCP). Its context receipt estimates
+text and reports total multimodal token cost as unknown. Encoded media is retained in history,
+but its bytes are not counted as text tokens. Text plus the finite output reserve must fit the
+requested window; Ollama determines the remaining model-specific fit with truncation and shifting
+disabled. A passing estimate alone does not prove the complete multimodal input fits.
+
+Scopes are process-local. Restarting `serve` discards them. Scope identifiers are handles within one
+operator service, not tenant authorization credentials. Apply external tenant isolation where
+required; see [Production deployment](PRODUCTION.md).
+
+Scope failures retain a stable code:
+
+| Status | Code | Meaning |
+|---|---|---|
+| `404` | `scope_not_found` | Missing, deleted, or expired scope. |
+| `409` | `scope_revision_conflict` | The expected revision differs from the stored revision. |
+| `409` | `scope_busy` | Another append owns the scope; fork for parallel work. |
+| `413` | `scope_history_limit_exceeded` | Complete history exceeds a per-scope bound. |
+| `422` | `scope_reference_invalid` | Scope identifier and expected revision were not supplied together. |
+| `422` | `scope_limit_invalid` | A requested limit exceeds operator caps or is zero. |
+| `422` | `scope_embedding_unsupported` | An embedding task or input attempted to use history. |
+| `429` | `scope_capacity_exceeded` | The store has reached its count or aggregate-byte bound. |
+| `502` | `scope_response_invalid` | A completed valid assistant message was not returned. |
+
+A minimal CLI sequence creates a prefix and submits its first turn:
+
+```bash
+npx @octocodeai/freellama scope create \
+  --json '{"messages":[{"role":"system","content":"Answer concisely."}]}'
+npx @octocodeai/freellama task --scope-id SCOPE_ID --scope-revision 0 \
+  "Explain the previous instruction."
+```
+
+Replace `SCOPE_ID` with the identifier returned by `scope create`; use the returned revision for
+each subsequent task.
+
+## Scope configuration
+
+The runtime file accepts a `[scopes]` table. Matching `FREELLAMA_SCOPE_*` environment values take
+precedence. Inspect effective values and their sources with
+`config`; reload behavior follows [Runtime configuration](MONITORING.md#change-settings-without-a-restart).
+
+| Setting | Environment variable | Default | Meaning |
+|---|---|---:|---|
+| `max_count` | `FREELLAMA_SCOPE_MAX_COUNT` | `128` | Maximum retained scopes. |
+| `max_messages` | `FREELLAMA_SCOPE_MAX_MESSAGES` | `256` | Maximum messages in one scope. |
+| `max_bytes` | `FREELLAMA_SCOPE_MAX_BYTES` | `1048576` | Maximum serialized history bytes in one scope. |
+| `max_estimated_tokens` | `FREELLAMA_SCOPE_MAX_ESTIMATED_TOKENS` | `32768` | Maximum estimated tokens in one scope. |
+| `total_max_bytes` | `FREELLAMA_SCOPE_TOTAL_MAX_BYTES` | `16777216` | Aggregate retained history-byte budget. |
+| `ttl_seconds` | `FREELLAMA_SCOPE_TTL_SECONDS` | `3600` | Idle expiry since create, fork, or successful append; reads do not renew it. |
+
+Per-scope `limits` accepts `max_messages`, `max_bytes`, `max_estimated_tokens`, and `ttl_seconds`;
+MCP uses `maxMessages`, `maxBytes`, `maxEstimatedTokens`, and `ttlSeconds`. A request can lower these
+bounds within server policy. Finite safety bounds remain operator-owned. Lowering runtime caps
+does not summarize or trim stored transcripts; subsequent writes must satisfy the active caps.
+Expired entries are removed during store operations.
+
+## Warm API
+
+`POST /warm`, MCP `warm_model`, and CLI `warm` load an exact installed model with a matching
+non-embedding task profile. They do not accept prompts or messages, pull a model, or store conversation
+history. The default task profile is `completion`.
+
+| HTTP field | MCP field | Meaning |
+|---|---|---|
+| `model` | `model` | Required exact installed tag. |
+| `task` | `task` | Optional capability/profile selection. |
+| `context_tokens` | `contextTokens` | Context profile used for warming. |
+| `execution_preference` | `executionPreference` | Guarded preference within operator assignments. |
+| `min_placement_evidence` | `minPlacementEvidence` | Required configured or observed placement evidence. |
+| `priority` | `priority` | Admission scheduling class. |
+| `keep_alive` | `keepAlive` | Explicit residency control; overrides adaptive selection. |
+| `max_wait_seconds` | `maxWaitSeconds` | Shorter admission/resource wait budget. |
+| `timeout_seconds` | `timeoutSeconds` | Total request deadline, capped by server policy. |
+| `defer` | `defer` | Return a job handle for the same managed operation. |
+
+Synchronous warming returns HTTP `200`; deferred warming returns HTTP `202`. Inspect deferred
+work through `task_jobs` or `jobs` using its returned ID. Warming reuses ordinary routing,
+admission, memory-fit checks, transition locking, deadlines, and physical-placement observations.
+A cold warm request can therefore wait or refuse. Read the `warm` receipt alongside
+`execution.observation` and `execution.keep_alive`; a successful load response alone does not prove
+physical residency or placement. Use configured placement for an initial load,
+inspect its receipt, and require observed placement for subsequent work when placement matters.
+
+`warm.loaded` reports observed residency independently of CPU/GPU placement: `true` when the exact
+runner is present, `false` when a successful residency query finds it absent, and `null` when that
+query is unavailable. A resident runner can still have a placement mismatch. Managed non-streaming
+chat requires Ollama's `done:true`; an absent or non-boolean completion marker is an upstream error.
+These are separate observations in Ollama's [chat response](https://docs.ollama.com/api/chat) and
+[running-model inventory](https://docs.ollama.com/api/ps).
+
+With `keep_alive:"0"`, warming observes placement and then unloads. The `warm` receipt describes
+residency after unload: `loaded:false` on verified unload, with
+`residency_source:"ollama_api_ps_after_unload"`.
+
+Warming uses the same truncation and context-shift policy as managed chat. This preserves a
+compatible runner profile instead of forcing a transition solely because warming omitted those
+controls. Use a context/task profile compatible with the following task. Warming a different profile can
+require another runner transition. Warm residency does not reserve a later execution slot. Cache reuse depends on the compatible
+profile, unchanged prefix, and inference backend. See [Managed execution](ARCHITECTURE.md#managed-task-execution).
+
+For a bounded warm operation, replace `MODEL_TAG` with an exact installed tag:
+
+```bash
+npx @octocodeai/freellama warm --model MODEL_TAG \
+  --context-tokens 4096 --keep-alive 60s --timeout-seconds 45 --defer
+```
+
+## Adaptive keep-alive
+
+When a managed request omits `keep_alive`, FreeLlama selects a finite duration from measured reuse
+and load cost, within the runtime `[warming]` bounds. Explicit per-call values take precedence,
+including immediate unload (`"0"`) and indefinite residency (`"-1"`). Use
+`--keep-alive=-1` in the CLI so the negative value remains attached to its flag. The configured eviction policy and available
+resources still govern whether retaining a runner is useful.
+
+Matching environment values override `[warming]` file values.
+
+| Setting | Environment variable | Default | Meaning |
+|---|---|---:|---|
+| `min_seconds` | `FREELLAMA_KEEP_ALIVE_MIN_SECONDS` | `15` | Minimum automatic duration. |
+| `max_seconds` | `FREELLAMA_KEEP_ALIVE_MAX_SECONDS` | `900` | Maximum automatic duration. |
+| `base_seconds` | `FREELLAMA_KEEP_ALIVE_BASE_SECONDS` | `60` | Base duration before measured reuse and load cost. |
+| `reuse_gain_seconds` | `FREELLAMA_KEEP_ALIVE_REUSE_GAIN_SECONDS` | `30` | Additional duration per recent use. |
+| `load_multiplier` | `FREELLAMA_KEEP_ALIVE_LOAD_MULTIPLIER` | `2.0` | Multiplier for measured load seconds. |
+| `pressure_factor` | `FREELLAMA_KEEP_ALIVE_PRESSURE_FACTOR` | `0.25` | Duration factor while observed resource pressure holds admission. |
+
+Automatic duration starts from `base_seconds + recent_uses × reuse_gain_seconds + load_seconds ×
+load_multiplier`. A pressure hold applies `pressure_factor`; minimum and maximum bounds clamp the
+result, rounded to whole seconds. Recent-use weight decays with the residency tracker; the load
+measurement comes from completed managed requests. These controls tune retention rather than model quality, CPU/GPU assignment, or host-memory
+reserves. Inspect `execution.keep_alive` policy evidence instead of inferring residency from a model name.
+
+Cold and warm helper-model measurements demonstrate feasibility on a specific host and request
+profile. They do not establish a universal speedup, justify lowering production reserves, or
+qualify sustained-load behavior. Compare representative cold, warm, scope-switching, and parallel
+work before changing policy; see [Testing](TESTING.md).

package/docs/TESTING.md

@@ -3,6 +3,10 @@
 FreeLlama separates fast deterministic checks from live-system tests. Run the narrowest relevant
 tier during development, then run the full matrix before release.
 
+The checks cover the management layer agents use to offload tasks: model eligibility, admission,
+context and runtime controls, resource receipts, and adapter behavior. Physical CPU/GPU activity
+and throughput require the separate hardware procedures linked in this guide.
+
 ```mermaid
 flowchart LR
     U["Unit and Rust contracts"] --> A["Local-agent context<br/>and action contracts"]
@@ -32,8 +36,9 @@ dependency rather than compiling Rust during installation.
 | JavaScript watch | `yarn test:watch` | None |
 | Rust contracts | `yarn test:rust` | Rust toolchain |
 | Local-agent contracts | `yarn test:agents` | Python 3 standard library |
+| Hardware runner contracts | `python3 -m unittest discover -s benchmark/hardware -p 'test_*.py'` | Python 3 standard library; no inference |
 | TypeScript | `yarn typecheck` | None after install |
-| MCP integration | `yarn test:integration` | Ollama on `127.0.0.1:11434` |
+| MCP integration | `yarn test:integration` | Ollama at `FREELLAMA_OLLAMA_ENDPOINT` (default `127.0.0.1:11434`) |
 | MCP end to end | `yarn test:e2e` | Ollama, `freellama serve`, and required models |
 | All configured tiers | `yarn test:all` | Requirements of every included tier |
 | Production verification | `yarn verify:production` | Build, formatting, strict Clippy, all test tiers, and package verification |
@@ -43,6 +48,26 @@ generation, embedding vectors are withheld by default, impossible models are exc
 and models with unusable research evidence are refused without tool calls. Tests that require a
 specific unavailable model report a skip reason.
 
+Isolated end-to-end services inherit `FREELLAMA_OLLAMA_ENDPOINT`; they use the same selected Ollama
+endpoint as the prerequisite check. A controlled endpoint fixture verifies this wiring without
+inference.
+
+Live inference tests also inspect routing readiness and report a skip when the host cannot admit
+the model. These skips do not qualify inference throughput or physical placement. The pull/delete
+lifecycle test is disabled unless the operator explicitly sets `FREELLAMA_TEST_PULL_TAG` to
+`qwen2.5:0.5b` and `FREELLAMA_TEST_PULL_SIZE_BYTES` to the exact reported download size. The test
+rechecks that size and reads every installed-model page before allowing the round trip.
+
+## Model metadata contracts
+
+Model metadata unit contracts cover README extraction, complete tag pages, family-versus-variant
+features, digest matching, cloud aliases, cache expiry and eviction, response bounds, and lookup
+failures. MCP integration's `model-knowledge.test.ts` exercises the built server with deterministic
+local HTTP fixtures for both Ollama API data and public HTML. It checks optional enrichment,
+assigned-backend inventory, tag pagination, validation, and preservation of benchmark/policy evidence.
+The fixture tests perform no inference, pulls, or deletions. See the
+[metadata reference](MODEL_METADATA.md) for the public states these tests protect.
+
 ## Run static Rust checks
 
 ```bash
@@ -52,6 +77,9 @@ cargo clippy --all-targets -- -D warnings
 
 Routing regressions run in the default Rust suite; none are hidden behind `#[ignore]`. Add a
 focused contract test for each repaired behavior and keep it enabled in ordinary release checks.
+Resource contracts cover conservative full host-memory reservations when discrete-GPU telemetry is
+missing, and immediate-unload cleanup after a failed inference response body. The latter also checks
+that uncertain inference is never replayed and ordinary keep-alive requests retain their runner.
 The local-agent tier runs all context fitting, compaction, pagination, repeat-suppression, and
 strict action-shape contracts. It is included in `yarn test:all`, so adapter regressions cannot be
 missed by the root release matrix.
@@ -71,6 +99,73 @@ two warm samples do not steer `auto`, the third sample on both backends enables
 comparison, differences of 10% or less remain noise, and one GPU plus one CPU request overlap even
 when both admission pools contain one unit.
 
+`resource_governor_contract` verifies that tasks held for memory remain counted in the bounded
+queue after releasing execution slots, resume when memory recovers, and release their queue entries
+and reservations on cancellation. A separate contract requires released reservations to wake
+waiters before the next telemetry sample. MCP integration checks forward `maxWaitSeconds` for
+single and batch execution and reject it in a decision-only preview.
+Admission contracts also cover expired deadlines, queue-limit reductions during resource-to-slot
+handoff, and cancellation after that handoff. Governor contracts require CPU pressure to recover
+independently while a memory hold still protects cold loads.
+MCP integration also requires native refusals to preserve error codes, retry timing, resource
+assessments, and unload receipts in both text and structured output.
+GET and DELETE refusals retain scope codes and job error details, while successful DELETE accepts an
+empty HTTP 204 body. Unknown top-level tool arguments are rejected before any server request;
+caller-owned message and function-definition extension fields remain preserved.
+
+`task_jobs_contract` covers deferred acceptance, metadata privacy, memory recovery, cancellation
+of waiting and active requests, removal of active and completed jobs without affecting other IDs,
+total deadlines during inference, and input-size/batch boundaries.
+Registry unit tests cover active-job limits, terminal eviction, expiry, and result-size limits.
+Cancellation must retain its receipt even when another operation evicts the completed record.
+`scope_warming_contract` covers opt-in history replay, message-field preservation, isolated forks,
+failed/stale append refusal, and payload-free managed warming. The warming unit contract
+exercises bounded reuse, load, and pressure adjustments. For request fields and failure codes, see
+[Scope history and model warming](SCOPES_AND_WARMING.md).
+`warming_receipt_contract` separates observed residency from processor verification, retains unknown
+residency when inspection fails, rejects incomplete chat responses, and verifies unload after failure.
+`resident_placement_contract` distinguishes installed file size from resident runner size across
+fresh discovery, cache refresh, observed routing, and scoped execution. Missing total runner memory
+with positive VRAM stays unknown and cannot qualify observed GPU routing or adaptive feedback.
+The MCP resident-view fixture checks percentages against resident bytes and preserves disk size.
+
+Usage-ledger unit contracts cover ordered writes, bounded queue overflow, failed writes, regular-file
+validation, replay, and flush behavior. `monitoring_contract` also checks that a blocked ledger path
+does not hold task completion. Persistence remains best effort, not a zero-loss promise.
+
+MCP `serve.test.ts` checks startup permission failure, retry, and child-lifecycle isolation. The
+protocol integration contracts verify that spawn failure leaves the MCP connection usable and that
+fixture-backed library requests fail on unexpected product errors. External catalog availability
+is separate from these deterministic checks. Runtime-view `app-windows.test.ts` checks legitimate
+Windows static paths alongside traversal refusal.
+
+Launcher fixtures use `process.execPath` to avoid an extra PATH-resolved shell wrapper. After the
+MCP fixture's real executable-permission refusal, a test-only transport launches that exact fixture
+script through the same Node executable. Real child processes, HTTP listeners, signals, and exit
+tracking still exercise the lifecycle contract. On the October 2, 2026 host, raw process samples
+placed the failed fixture launches in the macOS dynamic loader before JavaScript loaded. This
+fixture correction preserves test and production deadlines; it does not measure production
+startup performance or identify the underlying host loader policy.
+
+Request-validation contracts preserve structured JSON errors for invalid task and batch bodies.
+MCP `task-jobs.test.ts` exercises submission, inspection, result retrieval, cancellation, and removal through
+the native binding, plus preview rejection and embedding-vector summaries. Separate footprint and
+process-attribution tests require independent CPU/GPU settings; `smart_placement_contract` requires
+a resident routing preview to agree with execution under a low-memory-only hold.
+
+Run the isolated control timing experiments separately from the full suite:
+
+```bash
+cargo test --test control_timing_contract -- --include-ignored --nocapture
+cargo test --test task_jobs_contract measure_deferred_control_latency -- --ignored --nocapture
+```
+
+These fixed-budget experiments use simulated backends to measure status, discovery, queue
+controls, and resource recovery. They emit `CONTROL_TIMING` records with 20 samples, median, and
+95th-percentile latency. They verify both backend inventories and released reservations; they
+do not measure model tokens per second. Process-refresh contracts require updated observations
+after cache expiry and reject old footprint measurements after parallelism changes.
+
 ## Validate CPU and GPU concurrency
 
 Start the two Ollama processes and FreeLlama as described in
@@ -97,6 +192,22 @@ For promotion, use the portable [hardware acceptance runner](../benchmark/hardwa
 archive its JSON receipt with the release evidence. Compilation alone does not validate a driver,
 physical placement, shared-memory contention, or OCR quality.
 
+The runner measures separate sequential and parallel runs of identical workloads across at least
+three pairs, and checks every result before reporting successful-task throughput. It requires a
+configurable median gain (default 5%). Resident GPU bytes establish placement; they do not measure
+device busy percentage, bandwidth or energy efficiency. Sustained throughput and tail latency
+require a representative workload and separate device/thermal observations.
+
+The historical CPU/GPU observations above predate the corrected paired-run acceptance gate.
+They remain records of those runs, and do not qualify a release under the current gate.
+
+For a GPU workload with independent expected answers, use
+`benchmark/hardware/run_workload.py --workload MANIFEST.json --output RECEIPT.json`. It accepts
+distinct generation or vision cases, runs at least three alternating pairs, retains every
+result, and rejects incomplete or incorrect workloads. Its client deadline also bounds a hung
+measurement worker. See the [hardware guide](../benchmark/hardware/README.md) for the manifest
+contract.
+
 ## Verify release packages
 
 ```bash
@@ -114,6 +225,12 @@ prove registry publication or another hardware class.
 
 ## Test one layer while iterating
 
+The October 2, 2026 fixture investigation retained failed launches, raw dynamic-loader samples,
+and passing reruns. After the executable-fixture correction described above, the normal
+`yarn test` command passed 152 tests in 1.83 seconds of suite time. Test and production deadlines
+remain unchanged. Preserve failed and passing receipts together; this result does not qualify
+production startup timing.
+
 ```bash
 yarn build:native
 yarn workspace @octocodeai/freellama-mcp-server build