ollama-agent-router 0.1.5 → 0.1.7

package/README.md

@@ -1,9 +1,18 @@
 # ollama-agent-router
 
-`ollama-agent-router` is a local HTTP and CLI gateway for Ollama. It exposes an OpenAI-compatible chat completion endpoint and routes each request to the best configured local model based on task type, queue depth, loaded model state, GPU/VRAM headroom, priority, and sync/async policy.
+Ollama Agent Router is a local LLM router for Ollama. It provides an OpenAI-compatible API gateway that routes agent and chat requests to the best local Ollama model based on task type, GPU/VRAM headroom, queue depth, loaded model state, and sync/async policy.
 
 It is designed for machines that run several Ollama models with different strengths, for example a small triage model, one or more code models, and a larger exclusive reasoning model.
 
+## Why use Ollama Agent Router?
+
+Use `ollama-agent-router` when you need:
+
+- an Ollama router for multiple local models
+- an Ollama agent router for coding agents and autonomous workflows
+- an OpenAI-compatible local LLM router
+- GPU-aware routing, queues, async jobs, and model selection
+
 ## Architecture
 
 Request flow:
@@ -97,6 +106,7 @@ Lookup order:
 Top-level sections:
 
 - `server`: host, port, base path, HTTPS certificates, and JSON body limit.
+- `access`: optional access control for standalone, runtime-agent, and admin planes.
 - `ollama`: base URL, OpenAI-compatible path, native API path, keep-alive, timeout.
 - `gpu`: provider, VRAM limits, GPU-only default, NVIDIA monitor command.
 - `router`: default mode, heavy-load thresholds, classifier config.
@@ -116,6 +126,7 @@ Server options:
 
 ```yaml
 server:
+  nodeId: local
   host: 127.0.0.1
   port: 11435
   basePath: /
@@ -127,12 +138,13 @@ server:
     caPath:
 ```
 
-Set `server.port` to choose the listening port. Set `server.basePath` to expose every router endpoint under a prefix, for example `/ollama-router`; then chat completions move to `/ollama-router/v1/chat/completions`, health to `/ollama-router/health`, and jobs to `/ollama-router/v1/jobs/{jobId}`.
+Set `server.nodeId` to a stable machine/runtime id when the router is used behind Kong. It is embedded in new async job ids so a gateway can route job status/result requests back to the right node-router. Allowed characters are letters, numbers, dots, and dashes. Set `server.port` to choose the listening port. Set `server.basePath` to expose every router endpoint under a prefix, for example `/ollama-router`; then chat completions move to `/ollama-router/v1/chat/completions`, health to `/ollama-router/health`, and jobs to `/ollama-router/v1/jobs/{jobId}`.
 
 To run HTTPS directly from the router, set `server.https.enabled: true` and provide PEM certificate and key paths:
 
 ```yaml
 server:
+  nodeId: gex44-a
   host: 0.0.0.0
   port: 11435
   basePath: /ollama-router
@@ -144,6 +156,143 @@ server:
     caPath:
 ```
 
+## Access Planes
+
+The router can expose three separate access planes:
+
+- **Standalone plane**: the full local OpenAI-compatible router API, including `POST /v1/chat/completions` and job endpoints.
+- **Runtime agent plane**: machine-local endpoints used by Kong or another gateway, including `/v1/router/*` and selected-model execution.
+- **Admin plane**: access-management endpoints under `/v1/admin/access/*`.
+
+Access control is backward-compatible. If `access` is not configured, the standalone and runtime-agent planes stay enabled without API key requirements, matching earlier releases. The admin plane is disabled by default.
+
+API keys are sent with:
+
+```text
+Authorization: Bearer <api-key>
+```
+
+`x-api-key` is also accepted for clients that cannot set bearer tokens.
+
+Create SHA-256 key hashes before putting keys in config:
+
+```bash
+node -e "const crypto=require('crypto'); console.log('sha256:'+crypto.createHash('sha256').update(process.argv[1]).digest('hex'))" 'secret-value'
+```
+
+Example access configuration:
+
+```yaml
+access:
+  managedConfigPath: ./ollama-agent-router.access.yaml
+  bootstrapIfMissing: true
+
+  admin:
+    enabled: true
+    allowedIps: [127.0.0.1, "::1", 10.0.0.0/8]
+    trustedProxy: false
+    apiKeyHashes:
+      - sha256:replace-with-admin-key-hash
+    clientCert:
+      required: false
+      allowedFingerprints: []
+      allowedSubjects: []
+    auditLog: true
+
+  managed:
+    version: 1
+    planes:
+      standalone:
+        enabled: true
+        auth:
+          requireApiKey: true
+          anonymous: reject
+        defaultLimit:
+          requests: 60
+          windowSeconds: 60
+      runtimeAgent:
+        enabled: true
+        auth:
+          requireApiKey: true
+          anonymous: reject
+        defaultLimit:
+          requests: 600
+          windowSeconds: 60
+    apiKeys:
+      - id: local-client
+        name: Local standalone client
+        keyHash: sha256:replace-with-client-key-hash
+        enabled: true
+        scopes: [standalone]
+        limits:
+          standalone:
+            requests: 120
+            windowSeconds: 60
+      - id: kong-runtime
+        name: Kong runtime caller
+        keyHash: sha256:replace-with-kong-key-hash
+        enabled: true
+        scopes: [runtimeAgent]
+        limits:
+          runtimeAgent:
+            requests: 2000
+            windowSeconds: 60
+```
+
+`access.managed` is the initial access policy. When `access.managedConfigPath` is set, the router loads that file at startup. If the file is missing and `bootstrapIfMissing: true`, it writes the initial policy to that path. Admin API changes are then written atomically to this managed YAML file and survive restarts.
+
+The admin plane security settings are boot-only. They are intentionally not managed through the admin API:
+
+- `access.admin.allowedIps`
+- `access.admin.trustedProxy`
+- `access.admin.apiKeyHashes`
+- `access.admin.clientCert`
+
+This prevents the admin API from changing the rules that protect itself. When `access.admin.enabled: true`, `access.managedConfigPath` and at least one admin API key hash are required.
+
+Admin API:
+
+```bash
+curl http://127.0.0.1:11435/v1/admin/access/config \
+  -H 'authorization: Bearer admin-secret'
+
+curl -X PUT http://127.0.0.1:11435/v1/admin/access/config \
+  -H 'authorization: Bearer admin-secret' \
+  -H 'content-type: application/json' \
+  -d '{
+    "expectedVersion": 1,
+    "config": {
+      "version": 1,
+      "planes": {
+        "standalone": {
+          "enabled": true,
+          "auth": {"requireApiKey": true, "anonymous": "reject"},
+          "defaultLimit": {"requests": 60, "windowSeconds": 60}
+        },
+        "runtimeAgent": {
+          "enabled": true,
+          "auth": {"requireApiKey": true, "anonymous": "reject"},
+          "defaultLimit": {"requests": 600, "windowSeconds": 60}
+        }
+      },
+      "apiKeys": []
+    }
+  }'
+```
+
+The admin `PUT` supports optimistic concurrency. If `expectedVersion` is present and does not match the active managed access config, the router returns `409`.
+
+For admin client certificate checks, enable HTTPS, configure `server.https.caPath`, and set:
+
+```yaml
+access:
+  admin:
+    clientCert:
+      required: true
+```
+
+The HTTPS server requests a client certificate and the admin middleware verifies that it is trusted. Optional fingerprint and subject allowlists can narrow trust further.
+
 ## API Examples
 
 Sync-preferred request:
@@ -187,17 +336,68 @@ Status endpoints:
 curl http://127.0.0.1:11435/health
 curl http://127.0.0.1:11435/metrics
 curl http://127.0.0.1:11435/v1/router/status
+curl http://127.0.0.1:11435/v1/router/capabilities
+curl http://127.0.0.1:11435/v1/router/runtime
 curl http://127.0.0.1:11435/v1/router/models
 curl http://127.0.0.1:11435/v1/router/gpu
 ```
 
+## Kong Runtime Agent API
+
+When used with [`kong-ollama-agent-router`](https://github.com/ExeconOne/kong-ollama-agent-router), this process acts as a local runtime agent. Kong owns public request validation, classification, model selection, and response enrichment. The node-router supplies machine-local state and executes the model selected by Kong.
+
+Kong-facing endpoints:
+
+```bash
+curl http://127.0.0.1:11435/v1/router/capabilities
+curl http://127.0.0.1:11435/v1/router/runtime
+curl -X POST http://127.0.0.1:11435/v1/router/execute
+curl -X POST http://127.0.0.1:11435/v1/router/jobs
+```
+
+`GET /v1/router/capabilities` returns the stable routing config snapshot: `nodeId`, package version, router defaults, GPU policy, queue defaults, configured models, and routes. It does not call Ollama or GPU probes, so Kong can cache it for longer periods.
+
+`GET /v1/router/runtime` returns volatile runtime state: Ollama reachability, loaded models, GPU snapshot, queue depth/running counts, and retained job counters. Kong should cache it only briefly.
+
+`POST /v1/router/execute` runs a request on a model already selected by Kong. It does not classify or route again:
+
+```json
+{
+  "selectedModel": "deepseek-coder:6.7b",
+  "request": {
+    "model": "deepseek-coder:6.7b",
+    "messages": [{"role": "user", "content": "Review this TypeScript function"}],
+    "stream": false
+  },
+  "routerDecision": {
+    "taskType": "code_review",
+    "score": 250,
+    "reason": "Selected by Kong"
+  }
+}
+```
+
+The response is wrapped so Kong can add its own public `router` metadata:
+
+```json
+{
+  "result": {},
+  "nodeId": "gex44-a",
+  "selectedModel": "deepseek-coder:6.7b",
+  "queueTimeMs": 4,
+  "executionTimeMs": 1200
+}
+```
+
+`POST /v1/router/jobs` creates an async job on the selected model. New job ids include the node id, for example `job_gex44-a_01JABCDEF123`, so Kong can route later `GET /v1/jobs/{jobId}` and `GET /v1/jobs/{jobId}/result` calls to the owning node-router.
+
 ## Async Jobs
 
 When a selected model is busy or the router detects heavy load and `allowAsync=true`, the API returns:
 
 ```json
 {
-  "id": "job_01JABCDEF123",
+  "id": "job_gex44-a_01JABCDEF123",
   "object": "router.job",
   "status": "queued",
   "message": "Heavy load. Job accepted for asynchronous processing."
@@ -322,6 +522,7 @@ The project uses TypeScript, ESM, Express, zod, pino, p-queue, nanoid, and Vites
 Design notes:
 
 - CLI configuration wizard HLD: `docs/cli-configurator-hld.md`
+- Kong runtime agent contract plan: `docs/kong-runtime-contract-plan.md`
 
 ## Release Guide