dirsql-plugin-embeddings 0.1.12__tar.gz → 0.1.14__tar.gz

dirsql_plugin_embeddings-0.1.14/PKG-INFO

@@ -0,0 +1,102 @@
+Metadata-Version: 2.5
+Name: dirsql-plugin-embeddings
+Version: 0.1.14
+Summary: First-party dirsql plugin: semantic search over files.
+Requires-Python: >=3.10
+Requires-Dist: cachetta>=0.7
+Requires-Dist: click>=8
+Requires-Dist: dirsql
+Requires-Dist: model2vec>=0.6
+Requires-Dist: sqlite-vec>=0.1
+Requires-Dist: tqdm>=4
+Description-Content-Type: text/markdown
+
+# dirsql-plugin-embeddings
+
+A first-party [`dirsql`](https://github.com/thekevinscott/dirsql) plugin for
+semantic search over files.
+
+Installing the plugin loads the
+[`sqlite-vec`](https://github.com/asg017/sqlite-vec) extension (for
+`vec_distance_cosine()` and friends) and declares an `embed()` SQL scalar
+function that turns TEXT or BLOB values into embedding vectors:
+
+```sh
+uvx --with dirsql-plugin-embeddings dirsql "
+  SELECT path
+  FROM (SELECT path, embed(content ->> 'abstract') AS emb
+        FROM './arxiv-firehose/data/**/metadata.json')
+  ORDER BY vec_distance_cosine(emb, embed('local private models'))
+  LIMIT 10"
+```
+
+For the common case — one glob, one question, top-k paths — the package is
+also its own command, generating and running exactly that SQL:
+
+```sh
+uvx dirsql-plugin-embeddings '**/*.md' "local private models" -k 10
+```
+
+- **Corpus glob: required first positional.** The plugin never picks a
+  default corpus; you always say which files are in scope. A bare glob is
+  fine here — the command normalizes it to the `./`-relative form the SQL
+  layer requires (`**/*.md` → `./**/*.md`).
+- **Query text: second positional.** Query text, model id, and glob are
+  SQL-escaped into the generated query.
+- **`-k` / `--limit`** (both spellings, default 10): the number of results.
+  It is exactly the SQL `LIMIT` of the generated query — no other cutoff
+  exists.
+- **`--model <id>`**: templates the model id as `embed()`'s second argument
+  in the generated SQL (see [Model](#model)).
+
+Results print one `path<TAB>distance` line per match, closest first.
+
+> **Top-k is `LIMIT k`.** sqlite-vec's `MATCH ... AND k = N` idiom belongs to
+> its `vec0` virtual table, which `dirsql` does not use. For plain
+> expressions, sqlite-vec's own documented pattern is the one above:
+> `ORDER BY vec_distance_cosine(...) LIMIT k`.
+
+## Zero cost when unused
+
+`embed()` is inert until a query calls it: no worker process is spawned and
+no model is loaded for queries that never use it. On the first call, dirsql
+spawns the plugin's worker process (`dirsql-plugin-embeddings worker`), which
+serves every call of the invocation over stdin/stdout. Only the values the
+query actually selects are embedded — the worker receives values, not paths,
+and never opens files itself.
+
+## Model
+
+Embeddings come from [model2vec](https://github.com/MinishLab/model2vec)
+(static embeddings — numpy + tokenizers, no torch), defaulting to
+[`minishlab/potion-retrieval-32M`](https://huggingface.co/minishlab/potion-retrieval-32M).
+The model downloads to the standard Hugging Face cache on the first ever run
+(on the order of a hundred megabytes — seconds to a few minutes depending on
+your connection), with progress on stderr when stderr is a TTY; every later
+run loads it from disk.
+
+An optional second argument overrides the model per call — the id must be
+model2vec-loadable (sentence-transformers/torch models are out of scope):
+
+```sql
+SELECT embed('some text', 'minishlab/potion-base-8M')
+```
+
+The one-liner's `--model` flag templates the same second argument.
+
+## Vector cache
+
+Computed vectors are cached at `~/.cache/dirsql/embeddings/` (or
+`$XDG_CACHE_HOME/dirsql/embeddings/` when `XDG_CACHE_HOME` is set), keyed by
+the SHA-256 of the value bytes plus the model identifier — changing either
+recomputes; switching models never serves stale vectors. There is no
+eviction: **the directory is safe to wipe at any time**; the only cost is
+re-embedding. The cache never lives inside a queried tree — the worker
+receives values, not paths, and writes nothing anywhere else.
+
+## Docs
+
+- [Search documents by meaning](https://thekevinscott.github.io/dirsql/howto/search-by-meaning)
+  — the guide to both invocation styles.
+- [`[[dirsql.function]]`](https://thekevinscott.github.io/dirsql/reference/config#dirsql-function)
+  — the core mechanism `embed()` is built on.

dirsql_plugin_embeddings-0.1.14/README.md

@@ -0,0 +1,89 @@
+# dirsql-plugin-embeddings
+
+A first-party [`dirsql`](https://github.com/thekevinscott/dirsql) plugin for
+semantic search over files.
+
+Installing the plugin loads the
+[`sqlite-vec`](https://github.com/asg017/sqlite-vec) extension (for
+`vec_distance_cosine()` and friends) and declares an `embed()` SQL scalar
+function that turns TEXT or BLOB values into embedding vectors:
+
+```sh
+uvx --with dirsql-plugin-embeddings dirsql "
+  SELECT path
+  FROM (SELECT path, embed(content ->> 'abstract') AS emb
+        FROM './arxiv-firehose/data/**/metadata.json')
+  ORDER BY vec_distance_cosine(emb, embed('local private models'))
+  LIMIT 10"
+```
+
+For the common case — one glob, one question, top-k paths — the package is
+also its own command, generating and running exactly that SQL:
+
+```sh
+uvx dirsql-plugin-embeddings '**/*.md' "local private models" -k 10
+```
+
+- **Corpus glob: required first positional.** The plugin never picks a
+  default corpus; you always say which files are in scope. A bare glob is
+  fine here — the command normalizes it to the `./`-relative form the SQL
+  layer requires (`**/*.md` → `./**/*.md`).
+- **Query text: second positional.** Query text, model id, and glob are
+  SQL-escaped into the generated query.
+- **`-k` / `--limit`** (both spellings, default 10): the number of results.
+  It is exactly the SQL `LIMIT` of the generated query — no other cutoff
+  exists.
+- **`--model <id>`**: templates the model id as `embed()`'s second argument
+  in the generated SQL (see [Model](#model)).
+
+Results print one `path<TAB>distance` line per match, closest first.
+
+> **Top-k is `LIMIT k`.** sqlite-vec's `MATCH ... AND k = N` idiom belongs to
+> its `vec0` virtual table, which `dirsql` does not use. For plain
+> expressions, sqlite-vec's own documented pattern is the one above:
+> `ORDER BY vec_distance_cosine(...) LIMIT k`.
+
+## Zero cost when unused
+
+`embed()` is inert until a query calls it: no worker process is spawned and
+no model is loaded for queries that never use it. On the first call, dirsql
+spawns the plugin's worker process (`dirsql-plugin-embeddings worker`), which
+serves every call of the invocation over stdin/stdout. Only the values the
+query actually selects are embedded — the worker receives values, not paths,
+and never opens files itself.
+
+## Model
+
+Embeddings come from [model2vec](https://github.com/MinishLab/model2vec)
+(static embeddings — numpy + tokenizers, no torch), defaulting to
+[`minishlab/potion-retrieval-32M`](https://huggingface.co/minishlab/potion-retrieval-32M).
+The model downloads to the standard Hugging Face cache on the first ever run
+(on the order of a hundred megabytes — seconds to a few minutes depending on
+your connection), with progress on stderr when stderr is a TTY; every later
+run loads it from disk.
+
+An optional second argument overrides the model per call — the id must be
+model2vec-loadable (sentence-transformers/torch models are out of scope):
+
+```sql
+SELECT embed('some text', 'minishlab/potion-base-8M')
+```
+
+The one-liner's `--model` flag templates the same second argument.
+
+## Vector cache
+
+Computed vectors are cached at `~/.cache/dirsql/embeddings/` (or
+`$XDG_CACHE_HOME/dirsql/embeddings/` when `XDG_CACHE_HOME` is set), keyed by
+the SHA-256 of the value bytes plus the model identifier — changing either
+recomputes; switching models never serves stale vectors. There is no
+eviction: **the directory is safe to wipe at any time**; the only cost is
+re-embedding. The cache never lives inside a queried tree — the worker
+receives values, not paths, and writes nothing anywhere else.
+
+## Docs
+
+- [Search documents by meaning](https://thekevinscott.github.io/dirsql/howto/search-by-meaning)
+  — the guide to both invocation styles.
+- [`[[dirsql.function]]`](https://thekevinscott.github.io/dirsql/reference/config#dirsql-function)
+  — the core mechanism `embed()` is built on.

dirsql_plugin_embeddings-0.1.14/changelog.d/2026-08-12-no-per-call-progress-bar.md

@@ -0,0 +1,3 @@
+**Fixed**
+
+- **`embed()` no longer renders a progress bar for every call.** The worker embeds exactly one value per protocol round trip, so the per-call tqdm bar could only ever say `1/1` — on a TTY a 20-file corpus query printed ~21 of them to stderr. The bar is gone; the Hugging Face model-*download* bar, which measures something worth watching, is unchanged and still TTY-gated. (#814)

dirsql_plugin_embeddings-0.1.14/e2e-attestations/claude-814-suppress-per-call-progress.json

@@ -0,0 +1,7 @@
+{
+  "command": "uv run --with-editable ../../packages/python python -m pytest tests/e2e/ -x -q",
+  "ran_at": 1786552300,
+  "exit_code": 0,
+  "commit": "b1fc5a6a475a8f93eab7a3a652b5c982269768a8",
+  "branch": "claude/814-suppress-per-call-progress"
+}

{dirsql_plugin_embeddings-0.1.12 → dirsql_plugin_embeddings-0.1.14}/src/dirsql_plugin_embeddings/embedding/worker.py

@@ -3,7 +3,6 @@ from hashlib import sha256
 
 from . import model
 from .cache import make_cache
-from .progress import stderr_is_tty
 from .values import ProtocolError, decode_value
 
 MALFORMED_SHAPE = (
@@ -27,7 +26,10 @@ class Worker:
 
     def _compute(self, digest, identifier):
         text, loaded = self._pending
-        (vector,) = loaded.encode([text], show_progress_bar=stderr_is_tty())
+        # No progress bar: the protocol embeds one value per round trip, so a
+        # per-call bar can only ever say 1/1. The model *download* bar, which
+        # measures something, is TTY-gated in `progress.configure`.
+        (vector,) = loaded.encode([text], show_progress_bar=False)
         return [float(component) for component in vector]
 
     def embed(self, text, model_id):

dirsql_plugin_embeddings-0.1.12/PKG-INFO

@@ -1,61 +0,0 @@
-Metadata-Version: 2.5
-Name: dirsql-plugin-embeddings
-Version: 0.1.12
-Summary: First-party dirsql plugin: semantic search over files.
-Requires-Python: >=3.10
-Requires-Dist: cachetta>=0.7
-Requires-Dist: click>=8
-Requires-Dist: dirsql
-Requires-Dist: model2vec>=0.6
-Requires-Dist: sqlite-vec>=0.1
-Requires-Dist: tqdm>=4
-Description-Content-Type: text/markdown
-
-# dirsql-plugin-embeddings
-
-A first-party [`dirsql`](https://github.com/thekevinscott/dirsql) plugin for
-semantic search over files.
-
-Installing the plugin loads the
-[`sqlite-vec`](https://github.com/asg017/sqlite-vec) extension (for
-`vec_distance_cosine()` and friends) and declares an `embed()` SQL scalar
-function that turns TEXT or BLOB values into embedding vectors:
-
-```sh
-uvx --with dirsql-plugin-embeddings dirsql "
-  SELECT path
-  FROM (SELECT path, embed(content ->> 'abstract') AS emb
-        FROM 'arxiv-firehose/data/**/metadata.json')
-  ORDER BY vec_distance_cosine(emb, embed('local private models'))
-  LIMIT 10"
-```
-
-`embed()` is inert until a query calls it: no worker process is spawned and no
-model is loaded for queries that never use it. On the first call, dirsql
-spawns the plugin's worker process (`dirsql-plugin-embeddings worker`), which
-serves every call of the invocation over stdin/stdout.
-
-## Model
-
-Embeddings come from [model2vec](https://github.com/MinishLab/model2vec)
-(static embeddings — numpy + tokenizers, no torch), defaulting to
-[`minishlab/potion-retrieval-32M`](https://huggingface.co/minishlab/potion-retrieval-32M).
-The model downloads to the standard Hugging Face cache on the first ever run,
-with progress on stderr when stderr is a TTY.
-
-An optional second argument overrides the model per call — the id must be
-model2vec-loadable (sentence-transformers/torch models are out of scope):
-
-```sql
-SELECT embed('some text', 'minishlab/potion-base-8M')
-```
-
-## Vector cache
-
-Computed vectors are cached at `~/.cache/dirsql/embeddings/` (or
-`$XDG_CACHE_HOME/dirsql/embeddings/` when `XDG_CACHE_HOME` is set), keyed by
-the SHA-256 of the value bytes plus the model identifier — changing either
-recomputes; switching models never serves stale vectors. There is no
-eviction: **the directory is safe to wipe at any time**; the only cost is
-re-embedding. The cache never lives inside a queried tree — the worker
-receives values, not paths, and writes nothing anywhere else.

dirsql_plugin_embeddings-0.1.12/README.md

@@ -1,48 +0,0 @@
-# dirsql-plugin-embeddings
-
-A first-party [`dirsql`](https://github.com/thekevinscott/dirsql) plugin for
-semantic search over files.
-
-Installing the plugin loads the
-[`sqlite-vec`](https://github.com/asg017/sqlite-vec) extension (for
-`vec_distance_cosine()` and friends) and declares an `embed()` SQL scalar
-function that turns TEXT or BLOB values into embedding vectors:
-
-```sh
-uvx --with dirsql-plugin-embeddings dirsql "
-  SELECT path
-  FROM (SELECT path, embed(content ->> 'abstract') AS emb
-        FROM 'arxiv-firehose/data/**/metadata.json')
-  ORDER BY vec_distance_cosine(emb, embed('local private models'))
-  LIMIT 10"
-```
-
-`embed()` is inert until a query calls it: no worker process is spawned and no
-model is loaded for queries that never use it. On the first call, dirsql
-spawns the plugin's worker process (`dirsql-plugin-embeddings worker`), which
-serves every call of the invocation over stdin/stdout.
-
-## Model
-
-Embeddings come from [model2vec](https://github.com/MinishLab/model2vec)
-(static embeddings — numpy + tokenizers, no torch), defaulting to
-[`minishlab/potion-retrieval-32M`](https://huggingface.co/minishlab/potion-retrieval-32M).
-The model downloads to the standard Hugging Face cache on the first ever run,
-with progress on stderr when stderr is a TTY.
-
-An optional second argument overrides the model per call — the id must be
-model2vec-loadable (sentence-transformers/torch models are out of scope):
-
-```sql
-SELECT embed('some text', 'minishlab/potion-base-8M')
-```
-
-## Vector cache
-
-Computed vectors are cached at `~/.cache/dirsql/embeddings/` (or
-`$XDG_CACHE_HOME/dirsql/embeddings/` when `XDG_CACHE_HOME` is set), keyed by
-the SHA-256 of the value bytes plus the model identifier — changing either
-recomputes; switching models never serves stale vectors. There is no
-eviction: **the directory is safe to wipe at any time**; the only cost is
-re-embedding. The cache never lives inside a queried tree — the worker
-receives values, not paths, and writes nothing anywhere else.