@bendyline/gezel-knowledge 1.2.2 → 1.2.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -6
- package/dist/index.d.ts +510 -44
- package/dist/index.js +2118 -239
- package/package.json +4 -4
package/README.md
CHANGED
|
@@ -12,8 +12,9 @@ npm install @bendyline/gezel-knowledge
|
|
|
12
12
|
|
|
13
13
|
## What a `.gezk` is
|
|
14
14
|
|
|
15
|
-
An open format (format version 0.
|
|
16
|
-
archives still open
|
|
15
|
+
An open format (format version 0.8, preliminary until format 1.0; 0.5–0.7
|
|
16
|
+
archives still open, and the writer emits the oldest generation that can
|
|
17
|
+
express a catalog): a ZIP whose first entry is
|
|
17
18
|
the stored `mimetype` magic (`application/vnd.gezk+zip`), then
|
|
18
19
|
`manifest.json`, `README.md`, `LICENSES/`, a `router.db` (topics, document
|
|
19
20
|
directory, brotli bodies, routing centroids) and shard SQLite databases with
|
|
@@ -62,10 +63,19 @@ await compileKnowledgeCatalog({
|
|
|
62
63
|
|
|
63
64
|
Every archive ships a `README.md` and `LICENSES/catalog.txt`; the compiler
|
|
64
65
|
generates minimal ones from the catalog block unless you pass richer texts in
|
|
65
|
-
`extraFiles`. Embedding requires `@huggingface/transformers` at
|
|
66
|
-
module is imported dynamically so everything else works without
|
|
67
|
-
`gezel knowledge` CLI wraps this same API (`init`, `build`,
|
|
68
|
-
`inspect`, `search`), entirely offline — no daemon.
|
|
66
|
+
`extraFiles`. Embedding requires `@huggingface/transformers` (^4.3.1) at
|
|
67
|
+
runtime; the module is imported dynamically so everything else works without
|
|
68
|
+
it. The `gezel knowledge` CLI wraps this same API (`init`, `build`,
|
|
69
|
+
`validate`, `inspect`, `search`), entirely offline — no daemon.
|
|
70
|
+
|
|
71
|
+
With a multimodal profile (`EMBEDDINGGEMMA_2_512_1`), pass `embedMedia` too and
|
|
72
|
+
every image, video and audio file a document references becomes searchable
|
|
73
|
+
media rows (one per image, one per 30-second window), stored after the text
|
|
74
|
+
chunks so text chunk ids do not move. The compiler hands `embedMedia` the
|
|
75
|
+
asset bytes and its modality and expects raw model vectors back; it projects
|
|
76
|
+
them through the profile (`profileUnitVector`). `@bendyline/gezel-service/media`
|
|
77
|
+
exports a ready implementation (`createCatalogMediaEmbedder`) that decodes
|
|
78
|
+
images in pure JS and cuts video and audio with the system ffmpeg.
|
|
69
79
|
|
|
70
80
|
## Reading a catalog
|
|
71
81
|
|
|
@@ -86,6 +96,9 @@ Semantic search loads a shard's sign-bit rows into memory once (9.6 MB for a
|
|
|
86
96
|
full 200,000-chunk shard) and scores the float query against them through a
|
|
87
97
|
per-byte lookup table (the query is centered first when the catalog's profile
|
|
88
98
|
is `centered-sign`); the int8 rerank reads the top 1,024 candidates by rowid.
|
|
99
|
+
A query of the wrong width is refused with `CatalogQueryError('dimension')`.
|
|
100
|
+
Media rows (0.8) are scored exactly in their own lane, `searchMedia`, and
|
|
101
|
+
`assetFile` returns a declared asset's verified path for range streaming.
|
|
89
102
|
|
|
90
103
|
## Stability
|
|
91
104
|
|