@qvac/harness 0.8.0 → 0.8.6

package/CHANGELOG.md

@@ -8,6 +8,100 @@ While the major version is `0`, a breaking change bumps the minor version.
 
 ## [Unreleased]
 
+## [0.8.6] - 2026-10-08
+
+### Added
+
+- `setModelMemoryCap(percent)` lets loaded models fill that share of the
+  device's memory budget, so chat models a request loaded stay loaded beside
+  each other while they fit, and a chat switching between them loads neither
+  again. A load that would not fit unloads idle models one at a time: image and
+  music models first, then embedding, speech and transcription models, then
+  chat models a request loaded, then those loaded by hand, least recently used
+  first. Each load is sized by `assessModelFit`, and `list()` rows of loaded
+  models carry the estimate as `memoryBytes`; `memoryUsage()` answers their
+  total. A manual `loadModel` of any kind never unloads another model. `0`, a mobile device, and a host
+  that never sets a cap keep one chat model a request loaded, as before
+  (QVAC-25735). (#358)
+- `setDownloadsAllowed({ allowed })`, in process and over hrpc, lets a host hold
+  model downloads. While downloads are not allowed, a load whose model is not on
+  the device emits a `model` event with `status: 'held'` and the model's
+  download name and `addon`, then waits. It goes ahead once a `download()` puts
+  the model on the device or downloads are allowed again; aborting the caller's
+  signal ends it with `AbortedError`, and a wait that ends without loading
+  reports `error` with `message: 'aborted'`. This covers the image, music and translation
+  runtimes and every pooled load (chat, embedding, speech, transcription).
+  `download()` is never held, and downloads are allowed by default. Over hrpc,
+  every model event frame now carries an `addon` field, which is `null` unless
+  the event is `held` or the `error` that ends a held wait. (#340)
+- Ternary Bonsai 2 27B joins the curated catalog in two builds,
+  `Ternary-Bonsai-2-27B-PQ2_0` (7.2 GB) and `Ternary-Bonsai-2-27B-PTQ1_0`
+  (5.9 GB), each with its vision projection. Both sit in the `high` tier, by
+  their memory footprint rather than their parameter count. As a Qwen3.8-27B
+  fine-tune whose name the SDK's dialect router does not recognise, it parses
+  tool calls as `qwen35`; a catalog entry can now name its tool dialect. (#303)
+
+### Changed
+
+- `@qvac/sdk` and `@qvac/inference` 0.21.0, with `@qvac/llm-llamacpp` 0.55.0.
+  `@qvac/model-fit` leaves the native graph, and `@qvac/fabric` now ships its
+  prebuilds as per-platform packages (`@qvac/fabric-<os>-<cpu>`), so a host
+  that packages the addons exempts those names from any install age gate.
+  The fit check passes the model's source to the SDK's new `loadModel`-shaped
+  `assessModelFit` input. (#302)
+- Transcription streams long audio to the model in 300 s segments instead of
+  decoding the whole clip first, so peak memory no longer grows with the clip's
+  length (a 3.8 h clip peaked at about 0.9 GB above baseline, down from 1.8
+  GB). Clips under 60 s are transcribed as before. (#291)
+
+### Fixed
+
+- A run that preloads a skill (`@skill`, or a skill named in prose) caches its
+  prompt under a child of the chat's key for the length of the run, instead of
+  prefilling the whole prompt on every round of a tool loop. The child key is
+  deleted when the run ends. (#305)
+
+## [0.8.5] - 2026-10-06
+
+### Added
+
+- Skill-gated `pptx`, `docx` and `xlsx` tools on the office engine. A call takes
+  an `attachmentId` (or `create` options), `ops` that name engine methods with
+  named arguments, and an `output` file name; with an `attachmentId` alone it
+  returns the file's contents. The first failing op saves nothing and returns
+  the engine's error. An edit is never saved under the attached file's own
+  name. Saving the same file again in one turn answers with the one already
+  attached, and a changed file under that name is refused. The one-line answer
+  comes back as `reply`. These are the tools the presentations, word and excel
+  skills in `@qvac/skills` 0.1.12 call. (#286)
+
+### Changed
+
+- **Breaking:** `@qvac/skills` is now 0.1.12, whose office and pdf skills run on
+  the `pptx`, `docx`, `xlsx` and `pdf` tools instead of Python. A host that
+  passes its own `skillsDir` needs 0.1.12 as well: older office skills import
+  Python libraries this release no longer ships. (#286)
+- `extractOfficeText` and `extractDocxText` read .docx, .xlsx and .pptx in
+  process on the engine instead of starting Pyodide, and read at most 1M cells
+  across a workbook. Extracted workbook booleans read `true`/`false` instead of
+  `True`/`False`. `closeOfficeWorker` and `closeDocxWorker` remain as no-ops.
+  (#286)
+- `exec` refuses Python that imports `pptx`, `docx` or `openpyxl`, or declares
+  their packages, before it runs, and names the tool (or the skill that grants
+  it) to use instead. (#287)
+- **Breaking:** Python run through `exec` asks for approval like any other
+  write, including a run that only builds a new PDF. (#289)
+
+### Removed
+
+- **Breaking:** the vendored Python wheels for python-pptx, python-docx,
+  openpyxl, fpdf2 and pypdf, and their dependencies. Only micropip is still
+  seeded; any other package installs from PyPI, which sandbox mode refuses.
+  Use the office and `pdf` tools. (#286, #289)
+- The `exec` guidance written for python-pptx, python-docx and fpdf2 code:
+  their error hints and source checks, and the shrink-to-fit pass on saved
+  decks. (#288, #289)
+
 ## [0.8.0] - 2026-10-05
 
 ### Added
@@ -1108,7 +1202,9 @@ Tagged but never published to npm; these changes first shipped in 0.2.12.
 - GPU stats are read through `bare-gpu-info`, so every dependency installs from
   the public npm registry. (#12)
 
-[Unreleased]: https://github.com/tetherto/qvac-harness/compare/v0.8.0...HEAD
+[Unreleased]: https://github.com/tetherto/qvac-harness/compare/v0.8.6...HEAD
+[0.8.6]: https://github.com/tetherto/qvac-harness/compare/v0.8.5...v0.8.6
+[0.8.5]: https://github.com/tetherto/qvac-harness/compare/v0.8.0...v0.8.5
 [0.8.0]: https://github.com/tetherto/qvac-harness/compare/v0.7.3...v0.8.0
 [0.7.3]: https://github.com/tetherto/qvac-harness/compare/v0.7.2...v0.7.3
 [0.7.2]: https://github.com/tetherto/qvac-harness/compare/v0.7.1...v0.7.2

package/README.md

@@ -268,6 +268,55 @@ unloaded, and the next call loads what it needs again. The clock covers the
 whole device: any call resets it, whichever model it uses. `0`, the default, is
 Never.
 
+## Keeping several models in memory
+
+```js
+await harness.setModelMemoryCap(85)
+```
+
+Loaded models may fill that percent of the device's `memoryBudget`, so two chat
+models that fit stay loaded and a chat switching between them loads neither
+again. A model is unloaded only when the next load would not fit, one at a time:
+idle image and music models first, then embedding, speech and transcription
+models, then chat models a request loaded, then chat models loaded by hand, the
+least recently used first within each. A model a request holds stays, and a
+load past the cap goes ahead once nothing else can give way, so a model larger
+than the cap loads alone. A manual `loadModel`, of any kind, still never unloads
+another model: beside what is loaded it must fit the cap, or it is refused with
+`reason: 'memory'`. Lowering the cap unloads idle models past it at once. Loads
+under way count by their estimate, so two at once never count on the same room.
+
+Each load is sized by the SDK's `assessModelFit` for exactly the load it runs,
+falling back to the download size, and `list()` rows of loaded models carry that
+estimate as `memoryBytes`. `memoryUsage()` answers what every loaded model holds,
+whichever list rows a host reads. `0`, and any cap on a mobile device, keeps one chat
+model a request loaded, as without a cap; a host that never calls it keeps that
+behaviour and sizes nothing.
+
+## Holding model downloads
+
+```js
+await harness.setDownloadsAllowed({ allowed: false })
+```
+
+With downloads not allowed, a load whose model is not on the device does not
+download it. This covers every load the harness makes: chat, embedding, speech,
+transcription, image, music and translation. The load emits one `model` event
+with `status: 'held'`, the model's download name and its `addon`, then waits.
+Each `download()` that completes re-checks every waiting load, and a load goes
+ahead once its model is on the device, or once
+`setDownloadsAllowed({ allowed: true })` arrives, with no retry from the caller.
+Aborting the caller's signal ends the wait with `AbortedError`, and a suspend or
+close ends every wait. A wait that ends without loading reports `error` with
+`message: 'aborted'`. A released wait reports nothing of its own: the `loading`
+and `ready` after it name the model the way the pool does, which for embedding,
+speech and transcription is not the row name. So a host clears `held` when its
+`download()` of the row completes, when it allows downloads again, or on that
+`error`. A model already on the device loads with no wait. The
+check is the one `list({ cachedOnly: true })` makes, so a model counts as on the
+device only when its companions and projection are too. `download()` itself is
+never held. Downloads are allowed by default.
+
 ## Office extraction
 
 Hosts can supply the bundled Python extractor to their document pipeline:

package/dist-lib/index.d.ts

@@ -10,7 +10,7 @@ import { type EmbedContext } from './lib/capabilities/embed.ts';
 import { type KvCacheDeletionResult } from './lib/capabilities/kv-cache.ts';
 import type { LlmGenerationParams } from './lib/models/llm-presets.ts';
 import { type SpeakContext, type SpeakResult, type SpeakStreamContext, type TranscribeContext } from './lib/capabilities/voice.ts';
-import type { CapabilityInfo, CredentialEntry, HarnessDiagnostics, InfraModel, LoadModelResult, SkillsOptions, SkillWriteResult, UnloadModelContext, UnloadModelResult } from './lib/contract-types.ts';
+import type { CapabilityInfo, CredentialEntry, HarnessDiagnostics, InfraModel, LoadModelResult, MemoryUsage, SkillsOptions, SkillWriteResult, UnloadModelContext, UnloadModelResult } from './lib/contract-types.ts';
 import { type CustomModelEntry } from './lib/models/model-catalog.ts';
 import { type ListModelsOptions, type ModelInfo } from './lib/models/model-list.ts';
 import { type DeleteModelSummary } from './lib/models/model-delete.ts';
@@ -54,7 +54,7 @@ export { isAvailableState, SKILL_STATE, type SkillState } from './lib/tools/skil
 export { connectHarness, HARNESS_DIED, RemoteHarness } from './lib/connect.ts';
 export type { KvCacheDeletionResult } from './lib/capabilities/kv-cache.ts';
 export type { AgentKnobs, ApprovalCall, ChatMessage, CredentialEntry, DownloadContext, HarnessContract, HarnessDiagnostics, HarnessEvent, HarnessInfo, HarnessStatus, InfraModel, ModelState, SkillsOptions, SkillWriteResult } from './lib/contract-types.ts';
-import type { HarnessInfo } from './lib/contract-types.ts';
+import type { DownloadsAllowedOptions, HarnessInfo } from './lib/contract-types.ts';
 export { serveHarness } from './lib/serve.ts';
 export { hlog, setHarnessLog, setHarnessLogSink } from './lib/log.ts';
 export { harnessHooks } from './lib/hooks/registry.ts';
@@ -142,6 +142,8 @@ export declare class Harness extends ReadyResource {
     private readonly _pinned;
     private readonly _replying;
     private readonly _modelStatus;
+    private readonly _downloads;
+    private readonly _hold;
     private _models;
     private _embedModels;
     private readonly _retiredEmbedModels;
@@ -184,6 +186,11 @@ export declare class Harness extends ReadyResource {
     private _capabilityInflight;
     private readonly _modelFaults;
     private readonly _idle;
+    private _sizeLoads;
+    private _capBytes;
+    private readonly _reserved;
+    private readonly _admission;
+    private readonly _byHand;
     constructor({ tools, sdkLoader, skillsDir, modelsDir, projectionModel, imageModel, imageClamps, musicModel, musicClamps, llmDevice, llmGpuLayers, contextSize, kvCacheK, kvCacheV, embedDevice, embedGpuLayers, embedBatchSize, transcriptionDevice, completionModelCap, credentialStore, platform, gpuStats }?: HarnessOptions);
     _open(): Promise<void>;
     get suspended(): boolean;
@@ -213,6 +220,10 @@ export declare class Harness extends ReadyResource {
     private _projectionFor;
     setCustomModels(entries: ReadonlyArray<CustomModelEntry>): void;
     setModelIdleTimeout(ms: number): void;
+    setModelMemoryCap(percent: number): Promise<void>;
+    memoryUsage(): Promise<MemoryUsage>;
+    setDownloadsAllowed({ allowed }: DownloadsAllowedOptions): void;
+    private _isModelCached;
     previewModelSource(context: PreviewModelSourceContext): Promise<ModelSourcePreview>;
     private _assessFit;
     describe(): Promise<HarnessInfo>;
@@ -234,6 +245,7 @@ export declare class Harness extends ReadyResource {
     private _notifyModel;
     private _collectRetiredEmbedPool;
     private _releaseEmbedResource;
+    private _collectRetiredEmbedPoolAfter;
     private _releaseEmbedResourceAfter;
     private _destroyEmbedPools;
     private _enqueueEmbedControl;
@@ -275,6 +287,23 @@ export declare class Harness extends ReadyResource {
     private _loadAndRelease;
     private _unloadIdleGenerators;
     private _fits;
+    private _curatedFit;
+    private _admitByHand;
+    private _replacedBy;
+    private _sizeByHand;
+    private _loadByHand;
+    private _admit;
+    private _makeRoomForGenerator;
+    private _yieldGenerators;
+    private _beforeLoad;
+    private _reserve;
+    private _sizeLoad;
+    private _makeRoom;
+    private _capVictims;
+    private _victimRank;
+    private _usedBytes;
+    private _settledBytes;
+    private _memoryByModel;
     private _yieldPins;
     private _residency;
     private _completionLoadOptions;