@bendyline/gezel-knowledge 1.2.2 → 1.2.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -12,8 +12,9 @@ npm install @bendyline/gezel-knowledge
12
12
 
13
13
  ## What a `.gezk` is
14
14
 
15
- An open format (format version 0.6, preliminary until format 1.0; 0.5
16
- archives still open): a ZIP whose first entry is
15
+ An open format (format version 0.8, preliminary until format 1.0; 0.5–0.7
16
+ archives still open, and the writer emits the oldest generation that can
17
+ express a catalog): a ZIP whose first entry is
17
18
  the stored `mimetype` magic (`application/vnd.gezk+zip`), then
18
19
  `manifest.json`, `README.md`, `LICENSES/`, a `router.db` (topics, document
19
20
  directory, brotli bodies, routing centroids) and shard SQLite databases with
@@ -62,10 +63,19 @@ await compileKnowledgeCatalog({
62
63
 
63
64
  Every archive ships a `README.md` and `LICENSES/catalog.txt`; the compiler
64
65
  generates minimal ones from the catalog block unless you pass richer texts in
65
- `extraFiles`. Embedding requires `@huggingface/transformers` at runtime; the
66
- module is imported dynamically so everything else works without it. The
67
- `gezel knowledge` CLI wraps this same API (`init`, `build`, `validate`,
68
- `inspect`, `search`), entirely offline — no daemon.
66
+ `extraFiles`. Embedding requires `@huggingface/transformers` (^4.3.1) at
67
+ runtime; the module is imported dynamically so everything else works without
68
+ it. The `gezel knowledge` CLI wraps this same API (`init`, `build`,
69
+ `validate`, `inspect`, `search`), entirely offline — no daemon.
70
+
71
+ With a multimodal profile (`EMBEDDINGGEMMA_2_512_1`), pass `embedMedia` too and
72
+ every image, video and audio file a document references becomes searchable
73
+ media rows (one per image, one per 30-second window), stored after the text
74
+ chunks so text chunk ids do not move. The compiler hands `embedMedia` the
75
+ asset bytes and its modality and expects raw model vectors back; it projects
76
+ them through the profile (`profileUnitVector`). `@bendyline/gezel-service/media`
77
+ exports a ready implementation (`createCatalogMediaEmbedder`) that decodes
78
+ images in pure JS and cuts video and audio with the system ffmpeg.
69
79
 
70
80
  ## Reading a catalog
71
81
 
@@ -86,6 +96,9 @@ Semantic search loads a shard's sign-bit rows into memory once (9.6 MB for a
86
96
  full 200,000-chunk shard) and scores the float query against them through a
87
97
  per-byte lookup table (the query is centered first when the catalog's profile
88
98
  is `centered-sign`); the int8 rerank reads the top 1,024 candidates by rowid.
99
+ A query of the wrong width is refused with `CatalogQueryError('dimension')`.
100
+ Media rows (0.8) are scored exactly in their own lane, `searchMedia`, and
101
+ `assetFile` returns a declared asset's verified path for range streaming.
89
102
 
90
103
  ## Stability
91
104