@qvac/harness 0.5.0 → 0.6.0

package/CHANGELOG.md

@@ -8,6 +8,56 @@ While the major version is `0`, a breaking change bumps the minor version.
 
 ## [Unreleased]
 
+## [0.6.0] - 2026-09-28
+
+### Added
+
+- `loadModel({ model, agent })` loads a downloaded chat, embedding, speech or
+  transcription model ahead of use, under the key a request for it would load
+  under, so the first request finds it loaded. `unloadModel({ model })` unloads
+  every copy of a model once the requests holding it finish, and says whether it
+  could. Image and music models load for each generation and are refused. A load
+  or an unload counts as a call in flight for `setModelIdleTimeout`, and one that
+  lands during an idle unload waits for it. Also on the wire as `load-model` and
+  `unload-model`. (#133)
+
+  ```js
+  const { loaded } = await harness.loadModel({ model: QWEN_35_4B, agent: { ctxSize: 8192 } })
+  const { unloaded } = await harness.unloadModel({ model: QWEN_35_4B })
+  ```
+
+- A chat model loaded with `loadModel` is pinned: it does not count against
+  `completionModelCap`, so several stay loaded at once and a request for
+  another model leaves them in place. A manual load never unloads another model
+  to fit; it returns `{ loaded: false, reason: 'memory' }` when the SDK's
+  `assessModelFit` says it would not fit beside what is loaded. A request always
+  gets its model: pinned models no reply is using give way oldest first when it would not fit,
+  and a request or a manual load for a model under other settings replaces its idle copy. Image and
+  music generation and an idle unload still clear every idle chat model. (#134)
+
+  ```js
+  const { loaded, reason } = await harness.loadModel({ model: QWEN_35_9B })
+  if (reason === 'memory') showNotice('Unload a model first')
+  ```
+
+### Changed
+
+- A model loading or unloading in any pool emits `model` with `catalogChanged`,
+  not only a chat model, so `list()` reports a fresh `isLoaded` for embedding,
+  speech and transcription models too. (#133)
+- Speech, transcription and embedding requests key their pool entry by the
+  model's registry src however the caller names it, so a model named once by
+  its catalog name and once by its src is one copy in memory. (#133)
+
+### Fixed
+
+- A chat model loads when the host passes no `llmDevice`. 0.5.0 sent `device`,
+  `gpu_layers`, `main-gpu` and `openclCacheDir` to the SDK as `undefined`,
+  which the SDK spreads over its defaults: with no device the load failed with
+  "must specify a device: 'gpu' or 'cpu'", and with a device but no
+  `llmGpuLayers` the default of 99 GPU layers was lost. Unset fields are left
+  out again. (#137)
+
 ## [0.5.0] - 2026-09-25
 
 ### Added
@@ -840,7 +890,8 @@ Tagged but never published to npm; these changes first shipped in 0.2.12.
 - GPU stats are read through `bare-gpu-info`, so every dependency installs from
   the public npm registry. (#12)
 
-[Unreleased]: https://github.com/tetherto/qvac-harness/compare/v0.5.0...HEAD
+[Unreleased]: https://github.com/tetherto/qvac-harness/compare/v0.6.0...HEAD
+[0.6.0]: https://github.com/tetherto/qvac-harness/compare/v0.5.0...v0.6.0
 [0.5.0]: https://github.com/tetherto/qvac-harness/compare/v0.4.0...v0.5.0
 [0.4.0]: https://github.com/tetherto/qvac-harness/compare/v0.3.3...v0.4.0
 [0.3.3]: https://github.com/tetherto/qvac-harness/compare/v0.3.2...v0.3.3

package/README.md

@@ -193,6 +193,52 @@ user's files. The model stays registered and
 lists as not cached, ready to download again. Like a catalog delete, it does not
 unload a model that is running.
 
+### Load and unload a model
+
+```js
+import { QWEN_35_4B } from '@qvac/harness'
+
+const { loaded } = await harness.loadModel({ model: QWEN_35_4B, agent: { ctxSize: 8192 } })
+const row = (await harness.list({ cachedOnly: true })).find((model) => model.name === QWEN_35_4B)
+// row.isLoaded === true, and the next completion with the same agent reuses it
+
+const { unloaded } = await harness.unloadModel({ model: QWEN_35_4B })
+```
+
+`loadModel` loads a downloaded model ahead of use, under the key a request for it
+would load under, so the first request does not load it again. Pass the agent a
+chat model will run for, since its context size and KV cache types are part of
+that key. Chat, embedding, speech and transcription models stay loaded; image and
+music models load for each generation and are refused.
+
+A chat model loaded this way is pinned. It does not count against
+`completionModelCap`, so several can be loaded at once, and a request loading
+another model leaves it in place. A request or a manual load for the same model
+under other settings replaces its idle copy rather than loading a second one. A
+manual load never unloads another model to fit: `{ loaded: false, reason: 'memory' }` means the
+SDK's `assessModelFit` estimate says it would not fit beside what is loaded, so
+unload one first. A request always gets its model, though: when that would not
+fit, pinned models no reply is using give way, oldest first. Image or music generation and an idle
+unload still clear every idle chat model, pinned or not.
+
+```js
+const big = await harness.loadModel({ model: QWEN_36_27B, agent: { ctxSize: 262144 } })
+if (big.reason === 'memory') showNotice('Unload a model first')
+```
+
+`unloadModel` waits for any request holding the model to finish, then unloads
+every copy of it. `unloaded` is false when a request still held it after the
+wait, so a caller that wants the model gone at once stops those requests first.
+Both change `isLoaded` in the next `list()`, and emit a `model` event with
+`catalogChanged` set. A load or an unload counts as a call in flight for
+[`setModelIdleTimeout`](#freeing-memory-when-idle), and one that lands during an
+idle unload waits for it, so a load never loads a second copy and an unload
+answers once the memory is free.
+
+`bun run model-residency` (`bare examples/model-residency/run.ts`, with
+`QVAC_MODEL_CACHE_PATH` pointing at the models) runs these calls against the
+models already on disk and prints what each step changed. It never downloads.
+
 ## Freeing memory when idle
 
 ```js