@qvac/harness 0.7.0 → 0.7.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -8,6 +8,68 @@ While the major version is `0`, a breaking change bumps the minor version.
8
8
 
9
9
  ## [Unreleased]
10
10
 
11
+ ## [0.7.2] - 2026-10-01
12
+
13
+ ### Added
14
+
15
+ - `web_fetch` attaches a plain-text file (.txt, .csv, .json…) to the chat
16
+ besides returning its text, so the user can open or save it and `exec` can
17
+ stage it to read past the returned text. The result carries its
18
+ `attachmentId`, `fileName` and `byteLength`. A body cut at the fetch cap is
19
+ not attached, and a host that cannot store files gets the text alone, as
20
+ with page images. (#147, #148)
21
+
22
+ ### Changed
23
+
24
+ - Speech sounds cleaner on desktops. Devices on the `mid` profile and up now
25
+ speak with Supertonic 3 FP16 (`Supertonic3-FP16`, 207 MB) instead of Q8_0
26
+ (127 MB), at 16 inference steps instead of 5, which removes the artifacts
27
+ the 5-step output had. Synthesis takes about 2.7× as long, still around 7×
28
+ faster than real time on an M-series Mac. Phones (`low`) and `tiny` devices
29
+ keep Q4_0 at 5 steps. A host that prefetches `infraModelsFor` downloads the
30
+ FP16 model once; `Supertonic3-Q8_0` stays in the catalog, and now also runs
31
+ at 16 steps when a host asks for it by name. (#149)
32
+
33
+ ## [0.7.1] - 2026-09-30
34
+
35
+ ### Added
36
+
37
+ - `list()` rows carry `isLoading`: true from the moment a load of that model
38
+ starts, whoever asked for it, until it is ready or the load fails. It is read
39
+ off the pool, so a host shows a model as loading without counting loads
40
+ itself. A model loading again under other settings while a copy of it is
41
+ resident reads `isLoaded` and `isLoading` both true. Chat models read it
42
+ today: image, music and translation models load for each generation, and
43
+ speech, transcription and embedding pools key by source, so those rows read
44
+ `false` for now. An optional field on the wire; a row from an older harness
45
+ reads `false`. (#143)
46
+
47
+ ```js
48
+ const loading = (await harness.list({ cachedOnly: true })).filter((row) => row.isLoading)
49
+ ```
50
+
51
+ ### Changed
52
+
53
+ - The `model` event carries `catalogChanged` at every edge a `list()` re-read
54
+ would show: the `loading` that starts a load and the `ready` or `error` that
55
+ ends it, beside `unloaded`. Progress ticks still do not. A host that re-reads
56
+ the list on `catalogChanged` sees a model turn loading, then loaded, with no
57
+ bookkeeping of its own. The served event stream never drops a frame that
58
+ carries it, so two `ready` frames of one model under two keys both cross. (#143)
59
+ - `ModelInfo` requires `isLoading`, and `listModels`, `customModelRows` and
60
+ `mergeCustomRows` return `CatalogRow`, the row before the pools are asked. A
61
+ typed fake `list()` adds the field. (#143)
62
+ - A custom model's row reads `isLoaded` only once its load has landed; it used
63
+ to read `true` from the first attempt. (#143)
64
+
65
+ ### Fixed
66
+
67
+ - A load that ends without the model in memory emits `model` with `error` and
68
+ the reason: it failed, or a suspend or close cancelled it. It used to end in
69
+ silence, so a host could not tell a failed load from one still running. The
70
+ `error` also refreshes the cached `isCached`, since a load may have downloaded
71
+ the weights before it failed. (#143)
72
+
11
73
  ## [0.7.0] - 2026-09-29
12
74
 
13
75
  ### Added
@@ -979,7 +1041,9 @@ Tagged but never published to npm; these changes first shipped in 0.2.12.
979
1041
  - GPU stats are read through `bare-gpu-info`, so every dependency installs from
980
1042
  the public npm registry. (#12)
981
1043
 
982
- [Unreleased]: https://github.com/tetherto/qvac-harness/compare/v0.7.0...HEAD
1044
+ [Unreleased]: https://github.com/tetherto/qvac-harness/compare/v0.7.2...HEAD
1045
+ [0.7.2]: https://github.com/tetherto/qvac-harness/compare/v0.7.1...v0.7.2
1046
+ [0.7.1]: https://github.com/tetherto/qvac-harness/compare/v0.7.0...v0.7.1
983
1047
  [0.7.0]: https://github.com/tetherto/qvac-harness/compare/v0.6.0...v0.7.0
984
1048
  [0.6.0]: https://github.com/tetherto/qvac-harness/compare/v0.5.0...v0.6.0
985
1049
  [0.5.0]: https://github.com/tetherto/qvac-harness/compare/v0.4.0...v0.5.0
package/README.md CHANGED
@@ -165,6 +165,7 @@ const custom = models.filter((model) => model.registrySource === 'custom')
165
165
  // registrySource: 'custom',
166
166
  // isCached: false,
167
167
  // isLoaded: false,
168
+ // isLoading: false,
168
169
  // supportsImages: false,
169
170
  // expectedSize: 1868506272,
170
171
  // label: 'Nanbeige 4.2 3B',
@@ -234,8 +235,20 @@ if (big.reason === 'memory') showNotice('Unload a model first')
234
235
  `unloadModel` waits for any request holding the model to finish, then unloads
235
236
  every copy of it. `unloaded` is false when a request still held it after the
236
237
  wait, so a caller that wants the model gone at once stops those requests first.
237
- Both change `isLoaded` in the next `list()`, and emit a `model` event with
238
- `catalogChanged` set. A load or an unload counts as a call in flight for
238
+ Both change `isLoaded` in the next `list()`. `isLoading` is true on a row from
239
+ the moment a load of its model starts, whoever asked for it (a request,
240
+ `warmModel` or `loadModel`), until the model is ready or the load fails; it is
241
+ read off the pool, so a host keeps no count of its own. A model loading again
242
+ under other settings while a copy of it is resident reads `isLoaded: true` and
243
+ `isLoading: true` at once. The `model` event carries `catalogChanged` at each
244
+ edge a `list()` re-read would show: the `loading` that starts a load, the
245
+ `ready` or `error` that ends it, and `unloaded`. Progress ticks between them do
246
+ not. A load that ends without the model in memory reports
247
+ `error` with the reason in `message`: it failed, or a suspend or close cancelled it.
248
+ An image, music or translation model loads for each generation and never reads
249
+ `isLoading: true`; a speech, transcription or embedding model loads under its
250
+ source rather than its row name, so its row reads `false` for now. A load or an
251
+ unload counts as a call in flight for
239
252
  [`setModelIdleTimeout`](#freeing-memory-when-idle), and one that lands during an
240
253
  idle unload waits for it, so a load never loads a second copy and an unload
241
254
  answers once the memory is free.
@@ -218,6 +218,9 @@ export declare class Harness extends ReadyResource {
218
218
  describe(): Promise<HarnessInfo>;
219
219
  diagnostics(): HarnessDiagnostics;
220
220
  private _loadedModels;
221
+ private _loadingModels;
222
+ private _settledModels;
223
+ private _pools;
221
224
  private _loadPluginSdk;
222
225
  private _loadDiffusionSdk;
223
226
  private _slmTranslate;
package/dist-lib/index.js CHANGED
@@ -21,10 +21,10 @@ import { buildWavBuffer } from './lib/wav-encode.js';
21
21
  import { createVoiceLane, packSamples, speakOnce, speakStreamOnce, transcribeOnce, transcribeStreaming, SUPERTONIC_SAMPLE_RATE } from './lib/capabilities/voice.js';
22
22
  import { pushQueue } from './lib/queue.js';
23
23
  import { AbortedError, raceAbort } from './lib/helpers/race-abort.js';
24
- import { audioGenDownloadMetadata, customProjectionMap, defaultProjectionMap, defaultTtsFor, defaultVoiceModel, diffusionDownloadMetadata, curatedModelAsset, infraModelsFor, modelAddonOf, modelNameToCapabilityProfile, musicModelSet, residentSrc, setCustomModels } from './lib/models/model-catalog.js';
24
+ import { audioGenDownloadMetadata, categoryModelConfig, customProjectionMap, defaultProjectionMap, defaultTtsFor, defaultVoiceModel, diffusionDownloadMetadata, curatedModelAsset, infraModelsFor, modelAddonOf, modelNameToCapabilityProfile, musicModelSet, residentSrc, setCustomModels } from './lib/models/model-catalog.js';
25
25
  import { createModelStatusCache, customModelRows, invalidateModelStatusCache, mergeCustomRows, listModels } from './lib/models/model-list.js';
26
26
  import { deleteModelFiles, summarizeDelete } from './lib/models/model-delete.js';
27
- import { createEmbedPool, createLoadTracker, createModelPool, createVoicePool, awaitModelCapacity, evictIdle, evictIdleWhere, evictModel, evictModelWithin, isResident, leaseResidentModel, leaseCount, loadedModels, modelKey, modelOf, releaseModel, resolveLoadOptions, progressState } from './lib/models/model-pool.js';
27
+ import { createEmbedPool, createLoadTracker, createModelPool, createVoicePool, awaitModelCapacity, evictIdle, evictIdleWhere, evictModel, evictModelWithin, isResident, leaseResidentModel, leaseCount, loadedModels, loadingModels, settledModels, modelKey, modelOf, releaseModel, resolveLoadOptions, progressState } from './lib/models/model-pool.js';
28
28
  import { defaultContextSize } from './lib/models/context.js';
29
29
  import { resolveMainGpu } from './lib/models/gpu-selection.js';
30
30
  import { createIdleReaper } from './lib/models/idle-unload.js';
@@ -558,13 +558,22 @@ export class Harness extends ReadyResource {
558
558
  };
559
559
  }
560
560
  _loadedModels() {
561
- const pools = [
561
+ return this._pools().flatMap((pool) => loadedModels(pool));
562
+ }
563
+ _loadingModels() {
564
+ return new Set(this._pools().flatMap((pool) => loadingModels(pool)));
565
+ }
566
+ _settledModels() {
567
+ return new Set(this._pools().flatMap((pool) => settledModels(pool)));
568
+ }
569
+ _pools() {
570
+ return [
571
+ this._models,
562
572
  this._embedModels,
563
573
  ...this._retiredEmbedModels,
564
574
  this._ttsModels,
565
575
  this._transcriptionModels
566
576
  ];
567
- return [...loadedModels(this._models), ...pools.flatMap((pool) => loadedModels(pool))];
568
577
  }
569
578
  _loadPluginSdk(current, store, importPlugin) {
570
579
  let pending = current();
@@ -653,6 +662,7 @@ export class Harness extends ReadyResource {
653
662
  modelType: 'tts-ggml',
654
663
  addon: 'tts',
655
664
  modelConfig: TTS_MODEL_CONFIG,
665
+ modelConfigFor: categoryModelConfig,
656
666
  loads: this._loads
657
667
  });
658
668
  }
@@ -667,8 +677,7 @@ export class Harness extends ReadyResource {
667
677
  });
668
678
  }
669
679
  _notifyModel(state) {
670
- const catalogChanged = state.status === 'ready' || state.status === 'unloaded';
671
- if (catalogChanged) {
680
+ if (state.status === 'ready' || state.status === 'unloaded' || state.status === 'error') {
672
681
  invalidateModelStatusCache(this._modelStatus);
673
682
  }
674
683
  if (state.status === 'unloaded') {
@@ -678,7 +687,11 @@ export class Harness extends ReadyResource {
678
687
  }
679
688
  }
680
689
  }
681
- this.emit('model', { ...state, catalogChanged, loaded: this._loadedModels() });
690
+ this.emit('model', {
691
+ ...state,
692
+ catalogChanged: state.catalogChanged === true,
693
+ loaded: this._loadedModels()
694
+ });
682
695
  }
683
696
  async _collectRetiredEmbedPool(pool) {
684
697
  if (!this._retiredEmbedModels.has(pool) || pool.idle !== pool.size)
@@ -1055,17 +1068,17 @@ export class Harness extends ReadyResource {
1055
1068
  if (!this.opened)
1056
1069
  await this.ready();
1057
1070
  const sdk = await this._sdk.load();
1058
- const [rows, custom] = await Promise.all([
1059
- listModels(sdk, options, {
1060
- cache: this._modelStatus,
1061
- projectionFor: (name) => this._projectionFor(name)
1062
- }),
1063
- customModelRows(options, {
1064
- modelsDir: this._modelsDir,
1065
- loaded: new Set(this._loadedModels().map((entry) => entry.model))
1066
- })
1067
- ]);
1068
- return mergeCustomRows(rows, custom);
1071
+ const rows = await listModels(sdk, options, {
1072
+ cache: this._modelStatus,
1073
+ projectionFor: (name) => this._projectionFor(name)
1074
+ });
1075
+ const loaded = this._settledModels();
1076
+ const loading = this._loadingModels();
1077
+ const custom = await customModelRows(options, { modelsDir: this._modelsDir, loaded });
1078
+ return mergeCustomRows(rows, custom).map((row) => ({
1079
+ ...row,
1080
+ isLoading: loading.has(row.name)
1081
+ }));
1069
1082
  }
1070
1083
  async deleteModel(name) {
1071
1084
  if (!this.opened)
@@ -1,6 +1,7 @@
1
1
  import type { ModelRegistryEntry, ToolDialect } from '@qvac/sdk';
2
2
  import type { ImageModelSet } from '../tools/generate-image.ts';
3
3
  import type { MusicModelStages } from '../tools/generate-music.ts';
4
+ import type { ModelSrc } from './context.ts';
4
5
  import type { ModelAddon } from './model-source.ts';
5
6
  export interface SdkModelAsset extends ModelRegistryEntry {
6
7
  readonly src: string;
@@ -22,8 +23,10 @@ export interface CategoryDefault {
22
23
  readonly name: string;
23
24
  readonly category: ModelCategory;
24
25
  readonly asset: SdkModelAsset;
26
+ readonly modelConfig?: Readonly<Record<string, unknown>>;
25
27
  }
26
28
  export declare const CATEGORY_DEFAULTS: ReadonlyArray<CategoryDefault>;
29
+ export declare function categoryModelConfig(modelSrc: ModelSrc): Readonly<Record<string, unknown>> | undefined;
27
30
  export declare function defaultVoiceModel(category: ModelCategory): string;
28
31
  export declare function defaultTtsFor(profile: CapabilityProfile): CategoryDefault;
29
32
  export interface CatalogModel {
@@ -1,4 +1,4 @@
1
- import { AUDIOGEN_ACESTEP_5HZ_LM_0_6B_Q8_0, AUDIOGEN_ACESTEP_V15_SFT_Q8_0, AUDIOGEN_ACESTEP_V15_TURBO_Q4_K_M, AUDIOGEN_ACESTEP_V15_TURBO_Q8_0, AUDIOGEN_QWEN3_EMBEDDING_0_6B_Q8_0, AUDIOGEN_VAE_BF16, EMBEDDINGGEMMA_300M_Q4_0, FLUX_2_KLEIN_4B_Q4_0, FLUX_2_KLEIN_4B_VAE, HEALTHCARE_1_7B_MEDICAL_Q4_K_M, HEALTHCARE_4B_MEDICAL_Q4_K_M, MMPROJ_QWEN3_5_2B_MULTIMODAL_Q8_0, MMPROJ_QWEN3_5_4B_MULTIMODAL_Q8_0, MMPROJ_QWEN3_5_9B_MULTIMODAL_F16, MMPROJ_QWEN3_6_27B_MULTIMODAL_F16, MMPROJ_QWEN3_6_35B_A3B_MULTIMODAL_F16, MMPROJ_QWEN3_8_27B_MULTIMODAL_F16, QWEN3_5_2B_MULTIMODAL_Q4_K_M, QWEN3_5_4B_MULTIMODAL_Q4_K_M, QWEN3_5_9B_MULTIMODAL_Q4_K_M, PARAKEET_TDT_0_6B_V3_Q8_0, QWEN3_4B_Q4_K_M, QWEN3_6_27B_MULTIMODAL_Q4_K_XL, QWEN3_6_35B_A3B_MULTIMODAL_Q4_K_M, QWEN3_8_27B_MULTIMODAL_UD_Q4_K_XL, SD_V2_1_1B_Q8_0, SDXL_BASE_1_0_3B_Q4_0, TTS_MULTILINGUAL_SUPERTONIC3_Q4_0, TTS_MULTILINGUAL_SUPERTONIC3_Q8_0 } from '@qvac/sdk/models';
1
+ import { AUDIOGEN_ACESTEP_5HZ_LM_0_6B_Q8_0, AUDIOGEN_ACESTEP_V15_SFT_Q8_0, AUDIOGEN_ACESTEP_V15_TURBO_Q4_K_M, AUDIOGEN_ACESTEP_V15_TURBO_Q8_0, AUDIOGEN_QWEN3_EMBEDDING_0_6B_Q8_0, AUDIOGEN_VAE_BF16, EMBEDDINGGEMMA_300M_Q4_0, FLUX_2_KLEIN_4B_Q4_0, FLUX_2_KLEIN_4B_VAE, HEALTHCARE_1_7B_MEDICAL_Q4_K_M, HEALTHCARE_4B_MEDICAL_Q4_K_M, MMPROJ_QWEN3_5_2B_MULTIMODAL_Q8_0, MMPROJ_QWEN3_5_4B_MULTIMODAL_Q8_0, MMPROJ_QWEN3_5_9B_MULTIMODAL_F16, MMPROJ_QWEN3_6_27B_MULTIMODAL_F16, MMPROJ_QWEN3_6_35B_A3B_MULTIMODAL_F16, MMPROJ_QWEN3_8_27B_MULTIMODAL_F16, QWEN3_5_2B_MULTIMODAL_Q4_K_M, QWEN3_5_4B_MULTIMODAL_Q4_K_M, QWEN3_5_9B_MULTIMODAL_Q4_K_M, PARAKEET_TDT_0_6B_V3_Q8_0, QWEN3_4B_Q4_K_M, QWEN3_6_27B_MULTIMODAL_Q4_K_XL, QWEN3_6_35B_A3B_MULTIMODAL_Q4_K_M, QWEN3_8_27B_MULTIMODAL_UD_Q4_K_XL, SD_V2_1_1B_Q8_0, SDXL_BASE_1_0_3B_Q4_0, TTS_MULTILINGUAL_SUPERTONIC3_FP16, TTS_MULTILINGUAL_SUPERTONIC3_Q4_0, TTS_MULTILINGUAL_SUPERTONIC3_Q8_0 } from '@qvac/sdk/models';
2
2
  import { detectToolDialect } from './tool-dialect.js';
3
3
  import { defaultTtsModelName } from './tts-model-default.js';
4
4
  import { CAPABILITY_PROFILE_ORDER, QWEN_35_2B, QWEN_35_4B, QWEN_35_9B, QWEN_36_27B, QWEN_36_35B_A3B, QWEN_38_27B, MEDPSY_1_7B, MEDPSY_4B, isCapabilityProfile, meetsMinCapabilityProfile, modelNameToCapabilityProfile, capabilityProfileToModelName, isMultimodalLlm, AGENT_MODEL_NAMES, modelDisplayName, capabilityProfileModelLabel, DEFAULT_IMAGE_MODEL_CHOICE, IMAGE_CHOICE_CATALOG, isMobilePlatform, runnableModelName } from './model-catalog-constants.js';
@@ -62,9 +62,21 @@ export function defaultProjectionMap() {
62
62
  }
63
63
  return map;
64
64
  }
65
+ const TTS_FULL_STEPS = { ttsNumInferenceSteps: 16 };
65
66
  export const CATEGORY_DEFAULTS = [
66
67
  { name: 'Supertonic3-Q4_0', category: 'tts', asset: TTS_MULTILINGUAL_SUPERTONIC3_Q4_0 },
67
- { name: 'Supertonic3-Q8_0', category: 'tts', asset: TTS_MULTILINGUAL_SUPERTONIC3_Q8_0 },
68
+ {
69
+ name: 'Supertonic3-Q8_0',
70
+ category: 'tts',
71
+ asset: TTS_MULTILINGUAL_SUPERTONIC3_Q8_0,
72
+ modelConfig: TTS_FULL_STEPS
73
+ },
74
+ {
75
+ name: 'Supertonic3-FP16',
76
+ category: 'tts',
77
+ asset: TTS_MULTILINGUAL_SUPERTONIC3_FP16,
78
+ modelConfig: TTS_FULL_STEPS
79
+ },
68
80
  {
69
81
  name: 'Parakeet-TDT-0.6B-v3-Q8_0',
70
82
  category: 'transcription',
@@ -72,6 +84,10 @@ export const CATEGORY_DEFAULTS = [
72
84
  },
73
85
  { name: 'EmbeddingGemma-300M-Q4_0', category: 'embedding', asset: EMBEDDINGGEMMA_300M_Q4_0 }
74
86
  ];
87
+ export function categoryModelConfig(modelSrc) {
88
+ const src = typeof modelSrc === 'string' ? modelSrc : modelSrc.src;
89
+ return CATEGORY_DEFAULTS.find((entry) => entry.asset.src === src)?.modelConfig;
90
+ }
75
91
  export function defaultVoiceModel(category) {
76
92
  const entry = CATEGORY_DEFAULTS.find((candidate) => candidate.category === category);
77
93
  if (!entry) {
@@ -5,13 +5,17 @@ export interface ModelResource {
5
5
  }
6
6
  export interface ModelHandleOptions {
7
7
  load(): Promise<ModelResource>;
8
+ onLoaded?(): void;
9
+ onFailed?(err: unknown): void;
8
10
  }
9
11
  export declare class ModelHandle {
10
12
  private readonly _load;
11
13
  private readonly _peek;
14
+ private readonly _onLoaded;
15
+ private readonly _onFailed;
12
16
  private _tracked;
13
17
  private _loading;
14
- constructor({ load }: ModelHandleOptions);
18
+ constructor({ load, onLoaded, onFailed }: ModelHandleOptions);
15
19
  load(): Promise<ModelResource>;
16
20
  get loading(): boolean;
17
21
  peek(): Promise<ModelResource> | null;
@@ -2,12 +2,16 @@ import { retryMemo } from '../helpers/retry-memo.js';
2
2
  export class ModelHandle {
3
3
  _load;
4
4
  _peek;
5
+ _onLoaded;
6
+ _onFailed;
5
7
  _tracked = null;
6
8
  _loading = false;
7
- constructor({ load }) {
9
+ constructor({ load, onLoaded, onFailed }) {
8
10
  const memo = retryMemo(load);
9
11
  this._load = memo;
10
12
  this._peek = memo.peek;
13
+ this._onLoaded = onLoaded;
14
+ this._onFailed = onFailed;
11
15
  this.load().catch(noop);
12
16
  }
13
17
  load() {
@@ -15,12 +19,13 @@ export class ModelHandle {
15
19
  if (attempt !== this._tracked) {
16
20
  this._tracked = attempt;
17
21
  this._loading = true;
18
- const settled = () => {
22
+ const settle = (report) => {
19
23
  if (this._tracked === attempt) {
20
24
  this._loading = false;
25
+ report();
21
26
  }
22
27
  };
23
- attempt.then(settled, settled);
28
+ attempt.then(() => settle(() => this._onLoaded?.()), (err) => settle(() => this._onFailed?.(err)));
24
29
  }
25
30
  return attempt;
26
31
  }
@@ -16,12 +16,14 @@ export interface ModelInfo {
16
16
  registrySource: string;
17
17
  isCached: boolean;
18
18
  isLoaded: boolean;
19
+ isLoading: boolean;
19
20
  supportsImages: boolean;
20
21
  expectedSize: number;
21
22
  actualSize?: number;
22
23
  label?: string;
23
24
  taskType: ModelTaskType;
24
25
  }
26
+ export type CatalogRow = Omit<ModelInfo, 'isLoading'>;
25
27
  export interface ListModelsOptions {
26
28
  type?: ModelType;
27
29
  cachedOnly?: boolean;
@@ -45,11 +47,11 @@ export interface ListModelsRuntime {
45
47
  cache?: ModelStatusCache | undefined;
46
48
  projectionFor?: ((name: string) => string | undefined) | undefined;
47
49
  }
48
- export declare function listModels(sdk: ModelListSdk, { type, cachedOnly, fullRegistry }?: ListModelsOptions, { cache, projectionFor }?: ListModelsRuntime): Promise<ModelInfo[]>;
50
+ export declare function listModels(sdk: ModelListSdk, { type, cachedOnly, fullRegistry }?: ListModelsOptions, { cache, projectionFor }?: ListModelsRuntime): Promise<CatalogRow[]>;
49
51
  export interface CustomModelRowsContext {
50
52
  readonly modelsDir?: string;
51
53
  readonly loaded: ReadonlySet<string>;
52
54
  }
53
- export declare function customModelRows({ type, cachedOnly }: ListModelsOptions, { modelsDir, loaded }: CustomModelRowsContext): Promise<ModelInfo[]>;
54
- export declare function mergeCustomRows(catalog: ReadonlyArray<ModelInfo>, custom: ReadonlyArray<ModelInfo>): ModelInfo[];
55
+ export declare function customModelRows({ type, cachedOnly }: ListModelsOptions, { modelsDir, loaded }: CustomModelRowsContext): Promise<CatalogRow[]>;
56
+ export declare function mergeCustomRows(catalog: ReadonlyArray<CatalogRow>, custom: ReadonlyArray<CatalogRow>): CatalogRow[];
55
57
  export {};
@@ -5,8 +5,9 @@ import { type ModelAddon } from './model-source.ts';
5
5
  import type { GpuFaultState } from './sdk-faults.ts';
6
6
  type SdkModule = typeof import('@qvac/sdk');
7
7
  export interface ModelState {
8
- status: 'loading' | 'downloading' | 'downloaded' | 'ready' | 'unloading' | 'unloaded';
8
+ status: 'loading' | 'downloading' | 'downloaded' | 'ready' | 'error' | 'unloading' | 'unloaded';
9
9
  model: string;
10
+ catalogChanged?: boolean;
10
11
  [key: string]: unknown;
11
12
  }
12
13
  export type NotifyFn = (state: ModelState) => void;
@@ -78,6 +79,8 @@ type PoolLike = ModelPuddle;
78
79
  export declare function modelOf(key: string): string;
79
80
  export declare function releaseModel(pool: PoolLike, key: string): void;
80
81
  export declare function loadedModels(pool: PoolLike): LoadedModel[];
82
+ export declare function settledModels(pool: PoolLike): string[];
83
+ export declare function loadingModels(pool: PoolLike): string[];
81
84
  export interface ResidentLease {
82
85
  model: string;
83
86
  modelId: string;
@@ -153,8 +156,9 @@ export interface VoicePoolOptions {
153
156
  modelType: string;
154
157
  addon: ModelAddon;
155
158
  modelConfig?: Record<string, unknown> | undefined;
159
+ modelConfigFor?: (modelSrc: ModelSrc) => Readonly<Record<string, unknown>> | undefined;
156
160
  openclCacheDir?: (() => string | undefined) | undefined;
157
161
  loads?: LoadTracker;
158
162
  }
159
- export declare function createVoicePool(loadSdk: LoadVoiceSdk, notify: NotifyFn, { modelType, addon, modelConfig, openclCacheDir, loads }: VoicePoolOptions): ModelPuddle;
163
+ export declare function createVoicePool(loadSdk: LoadVoiceSdk, notify: NotifyFn, { modelType, addon, modelConfig, modelConfigFor, openclCacheDir, loads }: VoicePoolOptions): ModelPuddle;
160
164
  export {};
@@ -10,6 +10,7 @@ import { hlog } from '../log.js';
10
10
  import { RUNAWAY_STOP_SEQUENCES } from '../helpers/control-tokens.js';
11
11
  import { raceTimeout } from '../helpers/race-timeout.js';
12
12
  import { resolveModelSrc } from './model-source.js';
13
+ import { errorMessage } from '../guards.js';
13
14
  export function createLoadTracker() {
14
15
  const inflight = new Set();
15
16
  let draining = null;
@@ -101,6 +102,34 @@ export function loadedModels(pool) {
101
102
  }
102
103
  return out;
103
104
  }
105
+ export function settledModels(pool) {
106
+ const out = [];
107
+ for (const entry of pool.resources.values()) {
108
+ if (entry.value.peek() !== null && !entry.value.loading) {
109
+ out.push(modelOf(entry.id));
110
+ }
111
+ }
112
+ return out;
113
+ }
114
+ export function loadingModels(pool) {
115
+ const out = [];
116
+ for (const entry of pool.resources.values()) {
117
+ if (entry.value.loading) {
118
+ out.push(modelOf(entry.id));
119
+ }
120
+ }
121
+ return out;
122
+ }
123
+ function reportedHandle(notify, model, load) {
124
+ return new ModelHandle({
125
+ load: () => {
126
+ notify({ status: 'loading', model, catalogChanged: true });
127
+ return load();
128
+ },
129
+ onLoaded: () => notify({ status: 'ready', model, catalogChanged: true }),
130
+ onFailed: (err) => notify({ status: 'error', model, message: errorMessage(err), catalogChanged: true })
131
+ });
132
+ }
104
133
  export async function leaseResidentModel(pool, wanted = () => true) {
105
134
  for (const entry of pool.resources.values()) {
106
135
  const key = entry.id;
@@ -292,7 +321,6 @@ export function loadClockGuard(model, sdk, { now = () => Date.now(), monoMs = mo
292
321
  export function createModelPool(loadSdk, notify, { deviceConfig = {}, loads = createLoadTracker() } = {}) {
293
322
  const load = async (model, opts) => {
294
323
  const sdk = await loadSdk();
295
- notify({ status: 'loading', model });
296
324
  const modelSrc = await resolveModelSrc(sdk, model);
297
325
  const resolved = resolveLoadOptions(opts, deviceConfig);
298
326
  const ctxSize = resolved.ctxSize || defaultContextSize(modelSrc, customContextLength(model));
@@ -332,11 +360,10 @@ export function createModelPool(loadSdk, notify, { deviceConfig = {}, loads = cr
332
360
  onProgress: (progress) => notify(progressState(model, progress))
333
361
  })));
334
362
  hlog('llm loaded', { model, loadMs: Date.now() - loadStarted });
335
- notify({ status: 'ready', model });
336
363
  return { model, modelId, ctxSize };
337
364
  };
338
365
  return new ModelPuddle({
339
- open: (key, model = key, opts = {}) => new ModelHandle({ load: () => load(model, opts) }),
366
+ open: (key, model = key, opts = {}) => reportedHandle(notify, model, () => load(model, opts)),
340
367
  gc: unloadOnGc(loadSdk, notify),
341
368
  ...NO_IDLE_GC
342
369
  });
@@ -344,7 +371,6 @@ export function createModelPool(loadSdk, notify, { deviceConfig = {}, loads = cr
344
371
  export function createEmbedPool(loadSdk, notify, { faults, batchSize = DEFAULT_EMBED_BATCH_SIZE, openclCacheDir, loads = createLoadTracker() }) {
345
372
  const load = async (model) => {
346
373
  const sdk = await loadSdk();
347
- notify({ status: 'loading', model });
348
374
  const modelSrc = await resolveModelSrc(sdk, model, 'embeddings');
349
375
  const openclCache = faults.device !== 'cpu' && openclCacheDir ? openclCacheDir() : undefined;
350
376
  const modelId = await loadClockGuard(model, sdk).race(loads.track(sdk.loadModel({
@@ -358,24 +384,24 @@ export function createEmbedPool(loadSdk, notify, { faults, batchSize = DEFAULT_E
358
384
  },
359
385
  onProgress: (progress) => notify(progressState(model, progress))
360
386
  })));
361
- notify({ status: 'ready', model });
362
387
  return { model, modelId };
363
388
  };
364
389
  return new ModelPuddle({
365
- open: (key, model = key) => new ModelHandle({ load: () => load(model) }),
390
+ open: (key, model = key) => reportedHandle(notify, model, () => load(model)),
366
391
  gc: unloadOnGc(loadSdk, notify),
367
392
  ...NO_IDLE_GC
368
393
  });
369
394
  }
370
- export function createVoicePool(loadSdk, notify, { modelType, addon, modelConfig, openclCacheDir, loads = createLoadTracker() }) {
395
+ export function createVoicePool(loadSdk, notify, { modelType, addon, modelConfig, modelConfigFor, openclCacheDir, loads = createLoadTracker() }) {
371
396
  const wantsGpu = modelConfig?.['useGPU'] === true;
372
397
  let gpuRefused = false;
373
398
  const load = async (model) => {
374
399
  const sdk = await loadSdk();
375
- notify({ status: 'loading', model });
376
400
  const modelSrc = await resolveModelSrc(sdk, model, addon);
401
+ const tuned = modelConfigFor?.(modelSrc);
402
+ const loadConfig = tuned ? { ...modelConfig, ...tuned } : modelConfig;
377
403
  const attempt = () => {
378
- const config = gpuRefused ? { ...modelConfig, useGPU: false } : modelConfig;
404
+ const config = gpuRefused ? { ...loadConfig, useGPU: false } : loadConfig;
379
405
  const openclCache = gpuRefused ? undefined : openclCacheDir?.();
380
406
  return loads.track(sdk.loadModel({
381
407
  modelSrc,
@@ -391,11 +417,10 @@ export function createVoicePool(loadSdk, notify, { modelType, addon, modelConfig
391
417
  gpuRefused = true;
392
418
  return attempt();
393
419
  }));
394
- notify({ status: 'ready', model });
395
420
  return { model, modelId };
396
421
  };
397
422
  return new ModelPuddle({
398
- open: (key, model = key) => new ModelHandle({ load: () => load(model) }),
423
+ open: (key, model = key) => reportedHandle(notify, model, () => load(model)),
399
424
  gc: unloadOnGc(loadSdk, notify),
400
425
  ...NO_IDLE_GC
401
426
  });
@@ -423,7 +448,7 @@ function unloadOnGc(loadSdk, notify) {
423
448
  catch {
424
449
  continue;
425
450
  }
426
- notify({ status: 'unloaded', model });
451
+ notify({ status: 'unloaded', model, catalogChanged: true });
427
452
  }
428
453
  };
429
454
  }
@@ -1,9 +1,9 @@
1
1
  const TTS_MODEL_BY_PROFILE = {
2
2
  tiny: 'Supertonic3-Q4_0',
3
3
  low: 'Supertonic3-Q4_0',
4
- mid: 'Supertonic3-Q8_0',
5
- high: 'Supertonic3-Q8_0',
6
- xhigh: 'Supertonic3-Q8_0'
4
+ mid: 'Supertonic3-FP16',
5
+ high: 'Supertonic3-FP16',
6
+ xhigh: 'Supertonic3-FP16'
7
7
  };
8
8
  export const TTS_MODEL_NAMES = [
9
9
  ...new Set(Object.values(TTS_MODEL_BY_PROFILE))
@@ -270,7 +270,7 @@ function modelEvents(harness) {
270
270
  catalogChanged,
271
271
  ...loaded.map((entry) => `${entry.model}:${entry.refs}`)
272
272
  ].join('\0');
273
- if (key === last) {
273
+ if (key === last && !catalogChanged) {
274
274
  return;
275
275
  }
276
276
  last = key;
@@ -32,7 +32,8 @@ export function createWebFetchTool({ fetch: fetchImpl, resolveHost = lookupHost
32
32
  'shares, when a search snippet is not enough to answer. ' +
33
33
  'When the URL serves a file rather than a page — PDF, .docx, .xlsx, .pptx, an image, any ' +
34
34
  'binary — this is also the tool that downloads it: the bytes become a chat attachment and ' +
35
- 'the result carries its attachmentId. It is the only way to get a remote file into the ' +
35
+ 'the result carries its attachmentId. A plain-text file (.txt, .csv, .json…) is attached ' +
36
+ 'too, and its text is also returned. It is the only way to get a remote file into the ' +
36
37
  'chat; exec has no network and http_request truncates binary bodies.',
37
38
  parameters: {
38
39
  type: 'object',
@@ -133,13 +134,22 @@ export function createWebFetchTool({ fetch: fetchImpl, resolveHost = lookupHost
133
134
  const imagesLine = attached.length
134
135
  ? `\n\n[Attached ${attached.length === 1 ? 'image' : 'images'}: ${attached.join(', ')} — shown to the user automatically]`
135
136
  : '';
137
+ const file = isHtml
138
+ ? null
139
+ : await attachTextFile(response, { url: currentUrl, mimeType, attach: ctx?.attach });
140
+ const fileLine = file
141
+ ? `\n\n[Attached the file "${file.fileName}" (${Math.round(file.byteLength / 1024)}KB) — the user can open or save it; stage it with exec as inputs [{ "attachmentId": "${file.id}", "path": "${file.fileName}" }] to read past the text above]`
142
+ : '';
136
143
  const video = isHtml ? pageVideoRef(body, currentUrl) : null;
137
144
  const videoLine = video
138
145
  ? `\n\n[This page's video: ${video.url} — share this link when the user wants to watch it]`
139
146
  : '';
140
147
  const header = page.title ? `${page.title}\nSource: ${url}` : `Source: ${url}`;
141
148
  return {
142
- content: `${header}\n\n${text}${imagesLine}${videoLine}`,
149
+ attachmentId: file?.id,
150
+ fileName: file?.fileName,
151
+ byteLength: file?.byteLength,
152
+ content: `${header}\n\n${text}${fileLine}${imagesLine}${videoLine}`,
143
153
  data: {
144
154
  results: [
145
155
  {
@@ -201,6 +211,30 @@ async function attachDocument(response, { url, mimeType, attach }) {
201
211
  'user to attach a file this call already attached.'
202
212
  };
203
213
  }
214
+ async function attachTextFile(response, { url, mimeType, attach }) {
215
+ if (!attach)
216
+ return null;
217
+ const bytes = await response.bytes?.();
218
+ if (!bytes || bytes.byteLength === 0 || bytes.byteLength >= MAX_PAGE_BYTES) {
219
+ return null;
220
+ }
221
+ const mime = mimeType || 'text/plain';
222
+ try {
223
+ return await attach({
224
+ fileName: attachmentFileName(url, mime),
225
+ mimeType: mime,
226
+ bytes,
227
+ raw: true
228
+ });
229
+ }
230
+ catch {
231
+ return null;
232
+ }
233
+ }
234
+ const TEXT_EXTENSIONS = {
235
+ 'text/plain': 'txt',
236
+ 'text/markdown': 'md'
237
+ };
204
238
  const MIME_EXTENSIONS = {
205
239
  'application/pdf': 'pdf',
206
240
  'application/msword': 'doc',
@@ -234,7 +268,8 @@ function attachmentFileName(url, mimeType) {
234
268
  if (extension && acceptableExtensions(mimeType).has(extension)) {
235
269
  return base;
236
270
  }
237
- if (mimeType === 'application/octet-stream' && /^[a-z][\w-]*$/.test(extension)) {
271
+ const trustsUrl = mimeType === 'application/octet-stream' || !isBinaryMime(mimeType);
272
+ if (trustsUrl && /^[a-z][\w-]*$/.test(extension)) {
238
273
  return base;
239
274
  }
240
275
  return `${base}.${extensionFor(mimeType)}`;
@@ -248,7 +283,7 @@ function acceptableExtensions(mimeType) {
248
283
  return accepted;
249
284
  }
250
285
  function extensionFor(mimeType) {
251
- const known = MIME_EXTENSIONS[mimeType];
286
+ const known = MIME_EXTENSIONS[mimeType] ?? TEXT_EXTENSIONS[mimeType];
252
287
  if (known) {
253
288
  return known;
254
289
  }
@@ -339,7 +339,8 @@ export function registerHarnessTypes(ns) {
339
339
  { name: 'supportsImages', type: 'bool', required: true },
340
340
  { name: 'expectedSize', type: 'uint', required: true },
341
341
  { name: 'actualSize', type: 'uint' },
342
- { name: 'label', type: 'string' }
342
+ { name: 'label', type: 'string' },
343
+ { name: 'isLoading', type: 'bool' }
343
344
  ]
344
345
  });
345
346
  ns.register({
@@ -1,12 +1,12 @@
1
1
  // This file is autogenerated by the hyperschema compiler
2
- // Schema Version: 17
2
+ // Schema Version: 18
3
3
  /* eslint-disable camelcase */
4
4
  /* eslint-disable quotes */
5
5
  /* eslint-disable space-before-function-paren */
6
6
 
7
7
  import { c } from 'hyperschema/runtime'
8
8
 
9
- const VERSION = 17
9
+ const VERSION = 18
10
10
 
11
11
  // eslint-disable-next-line no-unused-vars
12
12
  let version = VERSION
@@ -1610,7 +1610,7 @@ const encoding31 = {
1610
1610
  c.string.preencode(state, m.type)
1611
1611
  c.string.preencode(state, m.registryPath)
1612
1612
  c.string.preencode(state, m.registrySource)
1613
- state.end++ // max flag is 16 so always one byte
1613
+ state.end++ // max flag is 32 so always one byte
1614
1614
  c.uint.preencode(state, m.expectedSize)
1615
1615
 
1616
1616
  if (m.actualSize) c.uint.preencode(state, m.actualSize)
@@ -1622,7 +1622,8 @@ const encoding31 = {
1622
1622
  (m.isLoaded ? 2 : 0) |
1623
1623
  (m.supportsImages ? 4 : 0) |
1624
1624
  (m.actualSize ? 8 : 0) |
1625
- ((version >= 15 && m.label) ? 16 : 0)
1625
+ ((version >= 15 && m.label) ? 16 : 0) |
1626
+ ((version >= 18 && m.isLoading) ? 32 : 0)
1626
1627
 
1627
1628
  c.string.encode(state, m.name)
1628
1629
  c.string.encode(state, m.type)
@@ -1651,7 +1652,8 @@ const encoding31 = {
1651
1652
  supportsImages: (flags & 4) !== 0,
1652
1653
  expectedSize: c.uint.decode(state),
1653
1654
  actualSize: (flags & 8) !== 0 ? c.uint.decode(state) : 0,
1654
- label: (version >= 15 && (flags & 16) !== 0) ? c.string.decode(state) : null
1655
+ label: (version >= 15 && (flags & 16) !== 0) ? c.string.decode(state) : null,
1656
+ isLoading: (version >= 18 && (flags & 32) !== 0)
1655
1657
  }
1656
1658
  }
1657
1659
  }
@@ -1,12 +1,12 @@
1
1
  // This file is autogenerated by the hyperschema compiler
2
- // Schema Version: 17
2
+ // Schema Version: 18
3
3
  /* eslint-disable camelcase */
4
4
  /* eslint-disable quotes */
5
5
  /* eslint-disable space-before-function-paren */
6
6
 
7
7
  import { c } from 'hyperschema/runtime'
8
8
 
9
- const VERSION = 17
9
+ const VERSION = 18
10
10
 
11
11
  // eslint-disable-next-line no-unused-vars
12
12
  let version = VERSION
@@ -1610,7 +1610,7 @@ const encoding31 = {
1610
1610
  c.string.preencode(state, m.type)
1611
1611
  c.string.preencode(state, m.registryPath)
1612
1612
  c.string.preencode(state, m.registrySource)
1613
- state.end++ // max flag is 16 so always one byte
1613
+ state.end++ // max flag is 32 so always one byte
1614
1614
  c.uint.preencode(state, m.expectedSize)
1615
1615
 
1616
1616
  if (m.actualSize) c.uint.preencode(state, m.actualSize)
@@ -1622,7 +1622,8 @@ const encoding31 = {
1622
1622
  (m.isLoaded ? 2 : 0) |
1623
1623
  (m.supportsImages ? 4 : 0) |
1624
1624
  (m.actualSize ? 8 : 0) |
1625
- ((version >= 15 && m.label) ? 16 : 0)
1625
+ ((version >= 15 && m.label) ? 16 : 0) |
1626
+ ((version >= 18 && m.isLoading) ? 32 : 0)
1626
1627
 
1627
1628
  c.string.encode(state, m.name)
1628
1629
  c.string.encode(state, m.type)
@@ -1651,7 +1652,8 @@ const encoding31 = {
1651
1652
  supportsImages: (flags & 4) !== 0,
1652
1653
  expectedSize: c.uint.decode(state),
1653
1654
  actualSize: (flags & 8) !== 0 ? c.uint.decode(state) : 0,
1654
- label: (version >= 15 && (flags & 16) !== 0) ? c.string.decode(state) : null
1655
+ label: (version >= 15 && (flags & 16) !== 0) ? c.string.decode(state) : null,
1656
+ isLoading: (version >= 18 && (flags & 32) !== 0)
1655
1657
  }
1656
1658
  }
1657
1659
  }
@@ -1,5 +1,5 @@
1
1
  {
2
- "version": 17,
2
+ "version": 18,
3
3
  "schema": [
4
4
  {
5
5
  "name": "empty",
@@ -1341,6 +1341,11 @@
1341
1341
  "name": "label",
1342
1342
  "type": "string",
1343
1343
  "version": 15
1344
+ },
1345
+ {
1346
+ "name": "isLoading",
1347
+ "type": "bool",
1348
+ "version": 18
1344
1349
  }
1345
1350
  ]
1346
1351
  },
@@ -1,5 +1,5 @@
1
1
  // This file is autogenerated by hyperschema-ts
2
- // Schema version: 17
2
+ // Schema version: 18
3
3
  // Do not edit manually.
4
4
 
5
5
  export interface HarnessEmpty {
@@ -257,6 +257,7 @@ export interface HarnessModelInfo {
257
257
  expectedSize: number
258
258
  actualSize?: number
259
259
  label?: string | null
260
+ isLoading?: boolean
260
261
  }
261
262
 
262
263
  export interface HarnessListModelsRequest {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@qvac/harness",
3
- "version": "0.7.0",
3
+ "version": "0.7.2",
4
4
  "type": "module",
5
5
  "engines": {
6
6
  "bare": ">=1.34.0"