llm-runtime-dock 0.1.0 → 0.1.2

package/README.md

@@ -222,6 +222,7 @@ field.
 server:
   host: 127.0.0.1
   port: 8787
+  idle_unload: 60m # unload after an hour of quiet; 0 to never
 
 runtimes:
   mtplx:
@@ -291,6 +292,14 @@ Clients send the logical id and never see the backend model:
 
 ### Fields you will actually set
 
+On the gateway itself:
+
+| field           | meaning                                                                   |
+| --------------- | ------------------------------------------------------------------------- |
+| `host` / `port` | where `lrd` listens. Defaults to `127.0.0.1:8787`                         |
+| `idle_unload`   | unload the loaded model after this long with nothing to do (below)        |
+| `auth`          | require an API key of clients, as `api_key_env` or `api_key_file` (below) |
+
 On a runtime:
 
 | field                        | meaning                                                                      |
@@ -335,7 +344,9 @@ Some arguments belong to the gateway and are rejected rather than silently
 merged: `--host`, `--port`, `--model`, served-id flags like `--identifier`,
 `--api-key`, and idle-unload flags such as LM Studio's `--ttl` — an idle
 auto-unload would drop the model behind the gateway's back and leave its view of
-the world wrong. The check covers `extra_args` too, and the error names the
+the world wrong. That is about _who owns the timer_, not about the idea: the
+gateway runs one itself, in front of its own bookkeeping, and you set it with
+`idle_unload` (below). The check covers `extra_args` too, and the error names the
 field to use instead. The full table, with the reasoning per flag, is in
 [§12](docs/05-configuration.md#configuration).
 
@@ -356,6 +367,51 @@ releasing foreign mtplx server on :8001 to free the resident slot
 
 The reasoning is in [§8 of the specification](docs/03-lifecycle.md#lifecycle).
 
+### Unloading when you stop using it
+
+`lrd serve` is a long-lived process, and until something asks for a different
+model nothing frees the one it is holding. Leave the gateway running after a
+morning's work and that 27B is still in memory at midnight.
+
+So it unloads by itself after an hour of quiet:
+
+```yaml
+server:
+  idle_unload: 60m # 0 never unloads
+```
+
+Write it as `60m`, `1h`, `90s`, `3600000ms`, or a bare number of milliseconds.
+`lrd serve --idle-unload 90s` overrides it for one run, and `lrd serve` prints
+the window it is using on startup:
+
+```text
+idle:     1h then unload
+```
+
+The next request loads the model again exactly as the first one did — the cost
+is one reload after an hour of not working, which is the trade the default is
+picked for. Turn it off with `0` if you would rather keep the model warm
+indefinitely.
+
+Two things it will not do:
+
+- **it never unloads a `keep_resident` model.** That flag means "for as long as
+  the gateway runs", and an idle spell is not the gateway stopping;
+- **it will not stop a server it did not start.** If `lrd` attached to an MTPLX
+  or custom server you launched yourself, stopping it is the only way to free
+  that memory — and doing that because `lrd` went quiet is not its call. It says
+  so in the log and leaves it alone. On LM Studio, oMLX and Ollama the question
+  never arises: unloading one model leaves the server, and everyone else on it,
+  untouched.
+
+When it has unloaded something, `lrd status` says so rather than just showing an
+empty slot:
+
+```text
+resident: none
+released: coding-quality (idle, stop_server)
+```
+
 ### Keeping one model always loaded
 
 Sometimes one model should never leave memory — a small one a coding agent
@@ -419,8 +475,8 @@ kept:      summariser (mtplx, ready, 0 active)
 serving:   summariser
 ```
 
-Its lifetime is the gateway's. Stopping `lrd serve` releases everything loaded,
-kept entries included — nothing would be left to free the memory otherwise, and
+Its lifetime is the gateway's — the idle unload above does not touch it either.
+Stopping `lrd serve` releases everything loaded, kept entries included — nothing would be left to free the memory otherwise, and
 a model outliving the process that loaded it is exactly the leak this gateway
 exists to prevent.
 
@@ -557,9 +613,11 @@ work because nothing rewrites them.
 Request bodies are touched in exactly one place: the `model` field, swapped from
 your logical id to the one the backend answers to. Not `tools`, not
 `tool_choice`, not `messages`. `GET /v1/models` returns your configured logical
-ids whether or not they are loaded, and your `Authorization` header goes
-upstream verbatim — a per-model `auth:` block fills in only when you sent none,
-and the gateway never generates a key.
+ids whether or not they are loaded, and — when the gateway itself requires no
+key — your `Authorization` header goes upstream verbatim; a per-model `auth:`
+block fills in only when you sent none. If `server.auth` is set (below), that
+header authenticates to the gateway instead and is not forwarded; the runtime's
+own `auth:` block is what reaches it, resolved server-side.
 
 The full surface, both protocols, response-header handling and `/switch`
 semantics are in [§14](docs/06-gateway-api.md#gateway-api).
@@ -623,9 +681,41 @@ the file it read.
 
 ## Security notes
 
-The gateway binds to `127.0.0.1` and has no authentication. `/status` and
-`/switch` are lifecycle controls: loopback-only, and refused outright when the
-gateway is bound to a non-loopback address.
+The gateway binds to `127.0.0.1` and requires no authentication by default.
+`/status` and `/switch` are lifecycle controls: loopback-only, refused outright
+when the gateway is bound to a non-loopback address, and that rule holds
+regardless of whether an API key is configured — a leaked key must not hand out
+remote process control on top of remote inference.
+
+**Requiring an API key.** Generate one and wire it into `server.auth`:
+
+```bash
+lrd key generate            # writes server.auth.api_key_file, plus a secret file next to the config
+lrd key generate --env      # prints the key once; writes server.auth.api_key_env: LRD_API_KEY instead
+```
+
+or set it by hand:
+
+```yaml
+server:
+  auth:
+    api_key_env: LRD_API_KEY # or: api_key_file: ~/.config/llm-runtime-dock/api_key
+```
+
+With no `server.auth` at all, the `LRD_API_KEY` environment variable is checked
+automatically — useful for a Docker or systemd deployment that already injects
+secrets that way, with no YAML edit needed. Once a key resolves, every route but
+`/health` requires it (`Authorization: Bearer <key>` or `x-api-key: <key>`);
+`lrd status`/`lrd switch` send it automatically, and `lrd apply` wires it into
+whichever agent you configure — Claude Code's `settings.json` gets the literal
+value (it has no reference syntax), OpenCode and Codex get an environment
+variable reference instead.
+
+An `api_key_env`/`api_key_file` that is set but resolves to nothing is a hard
+failure at `lrd serve`: an admin who configured a key source must never end up
+with a gateway that silently started unauthenticated. Binding to a non-loopback
+address with no key at all is only a warning, from `doctor` and the `serve`
+startup banner — not a hard stop.
 
 Lifecycle commands are **trusted configuration only**. Nothing derived from an
 HTTP request is ever interpolated into a command — a request selects which
@@ -651,8 +741,10 @@ metacharacter in it is live — quoting, globbing, redirection, command
 substitution. Use it only for a command you wrote yourself, and prefer the argv
 form, which cannot be reinterpreted.
 
-`lrd apply` never writes a credential into a coding agent's configuration, only
-an environment-variable reference.
+`lrd apply` never writes an _upstream_ credential into a coding agent's
+configuration, only an environment-variable reference — except the gateway's
+own key, which some agent formats (Claude Code) can only receive as a literal
+value, since that agent must present it to reach the gateway at all.
 
 The normative rules are in [§28](docs/01-overview.md#security).