@qvac/harness 0.5.0 → 0.7.0

package/CHANGELOG.md

@@ -8,6 +8,145 @@ While the major version is `0`, a breaking change bumps the minor version.
 
 ## [Unreleased]
 
+## [0.7.0] - 2026-09-29
+
+### Added
+
+- `list()` rows carry `taskType`, what the model is for: `llm`, `embeddings`,
+  `speech-to-text`, `text-to-speech`, `image-generation`, `audio-generation`,
+  `translation`, `text-recognition` or `other`. It is decided from the addon in
+  one place, `lib/models/model-task.ts`, on each side of the wire rather than
+  sent over it. `ModelTaskType` is exported, and grows the four new values a
+  preview never returns. (#140)
+
+- `modelsForAgent(agent, host)` lists every model an agent runs on a device,
+  each with its `use`: the chat model, clamped by `runnableModelName`, then the
+  image and music models its skills need. A host deciding what to download, or
+  which agents a model delete affects, asks this instead of working it out.
+  `runnableModelName(requested, host)` and `isMobilePlatform(platform)` move here
+  from `@qvac/core`: on a phone a pick above the device's profile runs as its
+  recommended model. Whether the host is a phone is the required `mobile` on
+  `HostCapability`, which `capability()` now reports, so its answer passes
+  straight in; a host built by hand has to say. Both functions are on the
+  browser-safe `@qvac/harness/model-catalog` entry too. (#141)
+
+  ```js
+  const chat = modelsForAgent(agent, await harness.capability()).find((m) => m.use === 'chat')
+  ```
+
+- `capability()` answers `mobile`, whether the harness runs on a phone, read off
+  the platform it probes. It rides `capability-response` as an optional field;
+  a response from an older harness reads as `false`. (#141)
+
+- `musicModelChoiceFrom(settings)` reads the music model pick out of an agent's
+  `generate_music` tool settings, so a host no longer knows what `modelChoice`
+  means there. `isMusicModelChoice` is exported beside it. (#141)
+
+### Changed
+
+- **Breaking:** the package no longer publishes a prebuilt sidecar:
+  `dist/harness.bundle`, `dist/harness.cjs`, `dist/prebuilds-manifest.json` and
+  the `./dist/harness.cjs` export are gone. A host stows
+  `worker/harness-sidecar.mjs` itself, as desktop and mobile already do, or
+  spawns `worker/stow-entry.mjs` straight from its install. (#127)
+- **Breaking:** a stowed harness runs python in a worker rather than in-process.
+  A host that stows the harness stops deferring `pyodide`, keeps
+  `QVAC_PYODIDE_CACHE_DIR` writable, no longer stages a separate worker bundle,
+  and ships what stow offloads beside the bundle, which now includes pyodide's
+  wasm, stdlib and lock file and the vendored wheels. A target that embeds
+  assets instead of offloading them cannot run python, since `fs` cannot read
+  inside a bundle. (#127)
+- A bundle of the package root carries neither pyodide nor the vendored wheels.
+  Only an entry that runs python, the exec tool or `@qvac/harness/docx` and
+  `/office`, pulls them in. (#127)
+- `SandboxHook` no longer checks `packages`. The exec tool refuses an install
+  the vendored wheels do not hold in full in sandbox mode, whether declared in
+  `packages` or on a pip line, with the same message as before. (#127)
+- The vendored wheels are the `#pyodide-wheels` directory asset on Bare;
+  `QVAC_PYTHON_WHEELS_DIR` still overrides it. (#127)
+- The python worker and the in-process deadline are spawned through
+  `worker_threads`, which the package maps to `bare-worker`;
+  `#pyodide-worker-spawn` is gone. (#127)
+- **Breaking:** `engines.bare` is `>=1.34.0`, so Bare refuses to load the
+  package on older runtimes. `bare-runtime` is pinned to 1.34.0 and
+  `bare-sidecar` to 0.5.7 (Bare 1.34.0) across the tree, `bare-supervisor`'s copy
+  included, with `bare-thread` at least 1.2.7 and `bare-module` 7.1.0 for
+  `bare-worker`'s copy. (#127)
+- `bare-module` is no longer a dependency or an override, and
+  `bare-module-traverse` is overridden to 2.5.8: earlier versions drop the
+  `asset` condition on `bare-sidecar`'s binary, and a stowed harness then fails
+  to boot on Bare 1.33 and later. (#127)
+- `skillModelsForAgent` rows carry `use`, `image` or `music`, so a caller can
+  tell which skill a model is for without looking the name up in the catalogs. (#141)
+- `CapabilityInfo` has a required `mobile`, so a `HarnessContract` whose
+  `capability()` builds its own answer, a test stub included, returns it too. (#141)
+
+### Removed
+
+- **Breaking:** `bareRuntime` is no longer exported from `./lib/tools/exec.ts`.
+  It is at `./lib/tools/exec/bare-runtime.ts`, so the package root no longer
+  loads the exec tool to reach it. (#127)
+- **Breaking:** `QVAC_PYODIDE_TEST_LOADER`: tests point
+  `QVAC_PYODIDE_WORKER_ENTRY` at an entry that serves their fake interpreter. (#127)
+
+### Fixed
+
+- Python runs start on Bare 1.34, whose threads cannot read a module off disk,
+  including in a stowed harness. The worker is spawned through `worker_threads`,
+  which the package maps to `bare-worker` on Bare, pyodide is handed its asm
+  module as `createPyodideModule`, and the files pyodide reads are named as
+  assets, so a bundler carries them beside the bundle. (#127)
+
+## [0.6.0] - 2026-09-28
+
+### Added
+
+- `loadModel({ model, agent })` loads a downloaded chat, embedding, speech or
+  transcription model ahead of use, under the key a request for it would load
+  under, so the first request finds it loaded. `unloadModel({ model })` unloads
+  every copy of a model once the requests holding it finish, and says whether it
+  could. Image and music models load for each generation and are refused. A load
+  or an unload counts as a call in flight for `setModelIdleTimeout`, and one that
+  lands during an idle unload waits for it. Also on the wire as `load-model` and
+  `unload-model`. (#133)
+
+  ```js
+  const { loaded } = await harness.loadModel({ model: QWEN_35_4B, agent: { ctxSize: 8192 } })
+  const { unloaded } = await harness.unloadModel({ model: QWEN_35_4B })
+  ```
+
+- A chat model loaded with `loadModel` is pinned: it does not count against
+  `completionModelCap`, so several stay loaded at once and a request for
+  another model leaves them in place. A manual load never unloads another model
+  to fit; it returns `{ loaded: false, reason: 'memory' }` when the SDK's
+  `assessModelFit` says it would not fit beside what is loaded. A request always
+  gets its model: pinned models no reply is using give way oldest first when it would not fit,
+  and a request or a manual load for a model under other settings replaces its idle copy. Image and
+  music generation and an idle unload still clear every idle chat model. (#134)
+
+  ```js
+  const { loaded, reason } = await harness.loadModel({ model: QWEN_35_9B })
+  if (reason === 'memory') showNotice('Unload a model first')
+  ```
+
+### Changed
+
+- A model loading or unloading in any pool emits `model` with `catalogChanged`,
+  not only a chat model, so `list()` reports a fresh `isLoaded` for embedding,
+  speech and transcription models too. (#133)
+- Speech, transcription and embedding requests key their pool entry by the
+  model's registry src however the caller names it, so a model named once by
+  its catalog name and once by its src is one copy in memory. (#133)
+
+### Fixed
+
+- A chat model loads when the host passes no `llmDevice`. 0.5.0 sent `device`,
+  `gpu_layers`, `main-gpu` and `openclCacheDir` to the SDK as `undefined`,
+  which the SDK spreads over its defaults: with no device the load failed with
+  "must specify a device: 'gpu' or 'cpu'", and with a device but no
+  `llmGpuLayers` the default of 99 GPU layers was lost. Unset fields are left
+  out again. (#137)
+
 ## [0.5.0] - 2026-09-25
 
 ### Added
@@ -840,7 +979,9 @@ Tagged but never published to npm; these changes first shipped in 0.2.12.
 - GPU stats are read through `bare-gpu-info`, so every dependency installs from
   the public npm registry. (#12)
 
-[Unreleased]: https://github.com/tetherto/qvac-harness/compare/v0.5.0...HEAD
+[Unreleased]: https://github.com/tetherto/qvac-harness/compare/v0.7.0...HEAD
+[0.7.0]: https://github.com/tetherto/qvac-harness/compare/v0.6.0...v0.7.0
+[0.6.0]: https://github.com/tetherto/qvac-harness/compare/v0.5.0...v0.6.0
 [0.5.0]: https://github.com/tetherto/qvac-harness/compare/v0.4.0...v0.5.0
 [0.4.0]: https://github.com/tetherto/qvac-harness/compare/v0.3.3...v0.4.0
 [0.3.3]: https://github.com/tetherto/qvac-harness/compare/v0.3.2...v0.3.3

package/README.md

@@ -12,41 +12,40 @@ the supervision contract.
 
 ## The published package
 
-The published package ships the prebuilt engine and its runtime content, not
-the library source:
-
-- `dist/harness.bundle` — the stowed sidecar (all platforms in one bundle)
-- `dist/prebuilds-manifest.json` — every native prebuild dir the bundle defers,
-  keyed by bundle-relative path; the host assembles those beside the bundle
-  from its own `node_modules`
+The published package ships the library and its runtime content, not a prebuilt
+engine:
+
+- `dist-lib/` — the compiled library and its declarations. The package exports
+  resolve to it; the TypeScript source stays in the repository.
+- `worker/harness-sidecar.mjs` — the sidecar entry a host stows into its own
+  bundle, and `worker/stow-entry.mjs`, the same entry as a plain file for a
+  host that spawns it straight from its install
 - the hash-pinned Python `wheels/` the tool runtime consumes (skills come from
   [`@qvac/skills`](https://github.com/tetherto/qvac-skills), a content-addressed
   bundle this package materializes at runtime)
 
-The `@qvac` inference addons and `pyodide` resolve from the host's real
-`node_modules` at runtime (dlopen'd backends need real paths), so the host
-installs those alongside.
-
-The host-side library surface — `spawnHarness`/`harnessChild`, the transport,
-model catalogs, skill helpers and DOCX extractor — ships as compiled JavaScript
-and declarations in `dist-lib/`. The package exports resolve to that surface;
-the TypeScript source stays in the repository.
+A host that stows the harness defers the `@qvac` inference addons
+(`scripts/addon-externals.cjs` lists them), so they resolve from real files
+beside the bundle, and ships everything stow offloads there: prebuilds, and
+pyodide's data and the wheels, which the harness names as assets. A target that
+embeds assets instead cannot run python.
 
 ## Spawning
 
-The generated `dist/harness.cjs` is the host entry: it spawns the bundle as a
-bare-sidecar process and hands back the stow ipc plus the exit promise.
+`spawnHarness` spawns an entry as a bare-sidecar process and hands back a
+client over hrpc:
 
 ```js
-const harness = require('@qvac/harness/dist/harness.cjs')
+import { spawnHarness } from '@qvac/harness'
 
-const { ipc, sidecar, exit } = await harness.start(args)
+const harness = spawnHarness({ entry, sdkStorage })
 ```
 
-The argv slots the bundle reads (skills dir, sdk storage, model config, wheels
-dir, ...) are the ones `worker/harness-sidecar.mjs` documents in this repo. The
-in-house hosts don't call this directly — they use the repo's library surface
-(`harnessChild` under bare-supervisor), which owns that argv contract.
+`entry` is the host's own stowed bundle, or `@qvac/harness/worker/stow-entry.mjs`
+resolved from its install. The argv slots the entry reads (skills dir, sdk
+storage, model config, wheels dir, ...) are the ones
+`worker/harness-sidecar.mjs` documents; `harnessChild` does the same under
+bare-supervisor.
 
 ## Custom models
 
@@ -168,7 +167,8 @@ const custom = models.filter((model) => model.registrySource === 'custom')
 //   isLoaded: false,
 //   supportsImages: false,
 //   expectedSize: 1868506272,
-//   label: 'Nanbeige 4.2 3B'
+//   label: 'Nanbeige 4.2 3B',
+//   taskType: 'llm'
 // }
 ```
 
@@ -181,6 +181,11 @@ sidecar knows from `sdkStorage.models`; an in-process `Harness` needs
 `new Harness({ modelsDir })` for it. A sharded download is looked for in the
 folder the SDK keeps its shards in, and is cached once every shard is there.
 
+Every row, catalog or custom, carries `taskType`: what the model is for, in the
+vocabulary a preview uses, so a Models screen groups by task rather than by
+addon. `whisper` and `parakeet` are both `speech-to-text`, and an addon with no
+task of its own is `other`.
+
 ### Delete a download
 
 ```js
@@ -193,6 +198,52 @@ user's files. The model stays registered and
 lists as not cached, ready to download again. Like a catalog delete, it does not
 unload a model that is running.
 
+### Load and unload a model
+
+```js
+import { QWEN_35_4B } from '@qvac/harness'
+
+const { loaded } = await harness.loadModel({ model: QWEN_35_4B, agent: { ctxSize: 8192 } })
+const row = (await harness.list({ cachedOnly: true })).find((model) => model.name === QWEN_35_4B)
+// row.isLoaded === true, and the next completion with the same agent reuses it
+
+const { unloaded } = await harness.unloadModel({ model: QWEN_35_4B })
+```
+
+`loadModel` loads a downloaded model ahead of use, under the key a request for it
+would load under, so the first request does not load it again. Pass the agent a
+chat model will run for, since its context size and KV cache types are part of
+that key. Chat, embedding, speech and transcription models stay loaded; image and
+music models load for each generation and are refused.
+
+A chat model loaded this way is pinned. It does not count against
+`completionModelCap`, so several can be loaded at once, and a request loading
+another model leaves it in place. A request or a manual load for the same model
+under other settings replaces its idle copy rather than loading a second one. A
+manual load never unloads another model to fit: `{ loaded: false, reason: 'memory' }` means the
+SDK's `assessModelFit` estimate says it would not fit beside what is loaded, so
+unload one first. A request always gets its model, though: when that would not
+fit, pinned models no reply is using give way, oldest first. Image or music generation and an idle
+unload still clear every idle chat model, pinned or not.
+
+```js
+const big = await harness.loadModel({ model: QWEN_36_27B, agent: { ctxSize: 262144 } })
+if (big.reason === 'memory') showNotice('Unload a model first')
+```
+
+`unloadModel` waits for any request holding the model to finish, then unloads
+every copy of it. `unloaded` is false when a request still held it after the
+wait, so a caller that wants the model gone at once stops those requests first.
+Both change `isLoaded` in the next `list()`, and emit a `model` event with
+`catalogChanged` set. A load or an unload counts as a call in flight for
+[`setModelIdleTimeout`](#freeing-memory-when-idle), and one that lands during an
+idle unload waits for it, so a load never loads a second copy and an unload
+answers once the memory is free.
+
+`bun run model-residency` (`bare examples/model-residency/run.ts`, with
+`QVAC_MODEL_CACHE_PATH` pointing at the models) runs these calls against the
+models already on disk and prints what each step changed. It never downloads.
+
 ## Freeing memory when idle
 
 ```js
@@ -227,8 +278,8 @@ are not extracted. Legacy XLS/PPT formats are unsupported. File extensions selec
 format, with `mimeType` as a fallback when the name has no recognized Office extension.
 
 The extractor accepts up to 20 MiB and requires the shipped Python wheels. With the
-cache configured, extraction uses those wheels offline. Bundled hosts also set `QVAC_PYODIDE_WORKER_ENTRY` and
-`QVAC_PYTHON_WHEELS_DIR` to their staged worker and wheels paths.
+cache configured, extraction uses those wheels offline. A host that stows the harness ships them
+with what stow offloads beside the bundle; `QVAC_PYTHON_WHEELS_DIR` names another directory.
 Source checkouts prepare wheels with `bun run vendor:wheels`.
 
 The existing `extractDocxText` and `closeDocxWorker` exports from `@qvac/harness/docx`
@@ -241,7 +292,7 @@ bun install
 bun run lint          # prettier + lunte + tsc
 bun run test          # brittle, node and bare lanes
 bun run build:schema  # regenerates spec/ (wire-compat rewind once v* tags exist)
-bun run build:bundle  # stows dist/harness.bundle (--minify for the shipped form)
+bun run build:bundle  # stows the sidecar entry into dist/, to prove it still stows
 ```
 
 The schema baseline rewinds `spec/` against this repo's `v*` release tags and

package/dist-lib/index.d.ts

@@ -10,7 +10,7 @@ import { type EmbedContext } from './lib/capabilities/embed.ts';
 import { type KvCacheDeletionResult } from './lib/capabilities/kv-cache.ts';
 import type { LlmGenerationParams } from './lib/models/llm-presets.ts';
 import { type SpeakContext, type SpeakResult, type SpeakStreamContext, type TranscribeContext } from './lib/capabilities/voice.ts';
-import type { CapabilityInfo, CredentialEntry, HarnessDiagnostics, InfraModel, SkillsOptions, SkillWriteResult } from './lib/contract-types.ts';
+import type { CapabilityInfo, CredentialEntry, HarnessDiagnostics, InfraModel, LoadModelResult, SkillsOptions, SkillWriteResult, UnloadModelContext, UnloadModelResult } from './lib/contract-types.ts';
 import { type CustomModelEntry } from './lib/models/model-catalog.ts';
 import { type ListModelsOptions, type ModelInfo } from './lib/models/model-list.ts';
 import { type DeleteModelSummary } from './lib/models/model-delete.ts';
@@ -38,6 +38,7 @@ export type { EmbedDevice } from './lib/models/embed-config.ts';
 export type { MemoryBudget } from './lib/hardware/memory-budget.ts';
 export * from './lib/models/model-catalog.ts';
 export * from './lib/models/model-list.ts';
+export type { ModelTaskType } from './lib/models/model-task.ts';
 export type { DeleteModelResult, DeleteModelSummary } from './lib/models/model-delete.ts';
 export { harnessChild, spawnHarness, type HarnessChildOptions, type SpawnHarnessOptions } from './lib/spawn.ts';
 export { settingsDimension, settingsSteps } from './lib/tools/generate-image.ts';
@@ -138,6 +139,8 @@ export declare class Harness extends ReadyResource {
     private _tools;
     private _sdk;
     private readonly _loads;
+    private readonly _pinned;
+    private readonly _replying;
     private readonly _modelStatus;
     private _models;
     private _embedModels;
@@ -211,6 +214,7 @@ export declare class Harness extends ReadyResource {
     setCustomModels(entries: ReadonlyArray<CustomModelEntry>): void;
     setModelIdleTimeout(ms: number): void;
     previewModelSource(context: PreviewModelSourceContext): Promise<ModelSourcePreview>;
+    private _assessFit;
     describe(): Promise<HarnessInfo>;
     diagnostics(): HarnessDiagnostics;
     private _loadedModels;
@@ -261,6 +265,16 @@ export declare class Harness extends ReadyResource {
     capability(): Promise<CapabilityInfo>;
     warmModel(context: WarmModelOptions): Promise<void>;
     private _warmModel;
+    loadModel(context: WarmModelOptions): Promise<LoadModelResult>;
+    private _loadModel;
+    unloadModel(context: UnloadModelContext): Promise<UnloadModelResult>;
+    private _unloadModel;
+    private _loadAndRelease;
+    private _unloadIdleGenerators;
+    private _fits;
+    private _yieldPins;
+    private _residency;
+    private _completionLoadOptions;
     invoke(context: InvokeContext): AsyncGenerator<CompletionEvent>;
     private _invoke;
     completion(context: CompletionContext): AsyncGenerator<CompletionEvent>;