veccore 0.2.0__tar.gz

veccore-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Ash Damle
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

veccore-0.2.0/PKG-INFO

@@ -0,0 +1,111 @@
+Metadata-Version: 2.4
+Name: veccore
+Version: 0.2.0
+Summary: Ports-and-adapters vector substrate: embed -> index -> search -> hydrate, backend/embedder-agnostic.
+Author: Ash Damle
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/ashdamle/keelworks
+Project-URL: Repository, https://github.com/ashdamle/keelworks
+Project-URL: Issues, https://github.com/ashdamle/keelworks/issues
+Keywords: embeddings,vector-search,qdrant,semantic-search,ports-and-adapters,retrieval
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.26
+Provides-Extra: qdrant
+Requires-Dist: qdrant-client>=1.9; extra == "qdrant"
+Provides-Extra: openai
+Requires-Dist: openai>=1.30; extra == "openai"
+Requires-Dist: tiktoken>=0.7; extra == "openai"
+Provides-Extra: st
+Requires-Dist: sentence-transformers>=2.7; extra == "st"
+Provides-Extra: fastembed
+Requires-Dist: fastembed>=0.3; extra == "fastembed"
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+Requires-Dist: pytest-timeout>=2.3; extra == "dev"
+Requires-Dist: build>=1.2; extra == "dev"
+Requires-Dist: twine>=5; extra == "dev"
+Dynamic: license-file
+
+# veccore
+
+**A ports-and-adapters vector substrate: `embed → index → search → hydrate`, backend- and embedder-agnostic.**
+
+veccore ships generic *mechanisms* and clean *seams* for semantic retrieval. You register a collection
+(choosing a backend + embedder), embed and upsert your items, and search — veccore returns
+`{id, score, payload}` and you hydrate the full record from your own system of record. Nothing about your
+domain lives in veccore; you never reinvent embedding, indexing, or idempotency.
+
+```bash
+pip install veccore                      # core (numpy only)
+pip install 'veccore[qdrant]'            # + Qdrant server backend
+pip install 'veccore[openai]'            # + OpenAI embedder (gated egress you inject)
+pip install 'veccore[fastembed]'         # + on-device fastembed embedder (no API key, no network)
+pip install 'veccore[st]'                # + sentence-transformers embedder
+```
+
+## Quickstart
+
+```python
+from veccore import (VecCore, CollectionRegistry, CollectionSpec, AuditEnvelope,
+                     MockEmbedder, SqliteNumpyStore)
+
+store = SqliteNumpyStore(path="/tmp/notes.sqlite")
+core = VecCore(registry=CollectionRegistry(),
+               stores={"sqlite_numpy": store},
+               embedders={"mock": MockEmbedder()})
+core.register(CollectionSpec(name="notes", backend="sqlite_numpy", embedder="mock", dim=64))
+
+# every call carries an audit envelope: who + why (where/what are filled by veccore)
+audit = AuditEnvelope(who="my_tool", why="index notes")
+core.embed_upsert("notes",
+                  [{"id": "n1", "text": "vector search over notes", "payload": {"src": "n1"}}],
+                  audit)
+
+hits = core.search("notes", "nearest ideas", k=5,
+                   audit=AuditEnvelope(who="my_tool", why="find related"))
+for h in hits:                           # each hit is {id, score, payload}
+    ...                                  # hydrate the full record from YOUR store by h.id
+```
+
+## What's in the box
+
+- **Core ports** — `EmbedderPort`, `VectorStorePort`, and `GatePort` (injected, so veccore stays free of
+  any spend/telemetry dependency — you hand it your gate, or `NullGate` for local/free embedders).
+- **Backends** — `SqliteNumpyStore` (in-process, brute-force cosine; implements the optional `matrix()`
+  analytics seam) and `QdrantStore` (server; implements `search(filters=…)` metadata/ACL push-down + Batch
+  ingest). Future backends slot in by implementing `VectorStorePort`.
+- **Embedders** — `MockEmbedder` (deterministic, zero-spend), `OpenAIEmbedder` (network, gated),
+  `LocalEmbedder` (fastembed / sentence-transformers, on-device), and `CompositeEmbedder` (fuse several
+  embedders so a plain dot product equals their weighted fused cosine).
+- **Content-sha idempotency** — re-embed a row only when its text changes; a rerun over an unchanged corpus
+  embeds nothing, and changing one item's text re-embeds exactly that one.
+- **Typed errors** — `Unavailable(code=…)` and `NotSupported(capability, backend)` — never a silent degrade.
+- **Estimate-first** — `core.estimate(...)` sizes a job (tokens + `$`) with zero spend before you embed.
+- **Batch ingest** — `BatchIngest` for large corpora via a provider Batch API: chunked, resumable, a durable
+  handle ledger, every result accounted; realtime and non-blocking collect both supported.
+- **Optional seams, negotiated** — `capabilities()` + typed `NotSupported` mean expanding usage is additive:
+  a consumer starts on `search(collection, q, k)`, later adds `filters=…`, later calls `matrix()`, with no
+  rewrite.
+
+## Design
+
+veccore's line is **mechanism vs policy**: veccore ships the generic mechanisms + seams; each consumer builds
+its domain policy on top. The gate (spend/egress) and any observability sink are **injected**, so veccore has
+one hard dependency — `numpy` — and everything else (Qdrant, OpenAI, fastembed, sentence-transformers) is an
+optional extra you install only for the adapter you choose.
+
+## License
+
+MIT © 2026 Ash Damle. Developed in the [keelworks](https://github.com/ashdamle/keelworks) monorepo.

veccore-0.2.0/README.md

@@ -0,0 +1,71 @@
+# veccore
+
+**A ports-and-adapters vector substrate: `embed → index → search → hydrate`, backend- and embedder-agnostic.**
+
+veccore ships generic *mechanisms* and clean *seams* for semantic retrieval. You register a collection
+(choosing a backend + embedder), embed and upsert your items, and search — veccore returns
+`{id, score, payload}` and you hydrate the full record from your own system of record. Nothing about your
+domain lives in veccore; you never reinvent embedding, indexing, or idempotency.
+
+```bash
+pip install veccore                      # core (numpy only)
+pip install 'veccore[qdrant]'            # + Qdrant server backend
+pip install 'veccore[openai]'            # + OpenAI embedder (gated egress you inject)
+pip install 'veccore[fastembed]'         # + on-device fastembed embedder (no API key, no network)
+pip install 'veccore[st]'                # + sentence-transformers embedder
+```
+
+## Quickstart
+
+```python
+from veccore import (VecCore, CollectionRegistry, CollectionSpec, AuditEnvelope,
+                     MockEmbedder, SqliteNumpyStore)
+
+store = SqliteNumpyStore(path="/tmp/notes.sqlite")
+core = VecCore(registry=CollectionRegistry(),
+               stores={"sqlite_numpy": store},
+               embedders={"mock": MockEmbedder()})
+core.register(CollectionSpec(name="notes", backend="sqlite_numpy", embedder="mock", dim=64))
+
+# every call carries an audit envelope: who + why (where/what are filled by veccore)
+audit = AuditEnvelope(who="my_tool", why="index notes")
+core.embed_upsert("notes",
+                  [{"id": "n1", "text": "vector search over notes", "payload": {"src": "n1"}}],
+                  audit)
+
+hits = core.search("notes", "nearest ideas", k=5,
+                   audit=AuditEnvelope(who="my_tool", why="find related"))
+for h in hits:                           # each hit is {id, score, payload}
+    ...                                  # hydrate the full record from YOUR store by h.id
+```
+
+## What's in the box
+
+- **Core ports** — `EmbedderPort`, `VectorStorePort`, and `GatePort` (injected, so veccore stays free of
+  any spend/telemetry dependency — you hand it your gate, or `NullGate` for local/free embedders).
+- **Backends** — `SqliteNumpyStore` (in-process, brute-force cosine; implements the optional `matrix()`
+  analytics seam) and `QdrantStore` (server; implements `search(filters=…)` metadata/ACL push-down + Batch
+  ingest). Future backends slot in by implementing `VectorStorePort`.
+- **Embedders** — `MockEmbedder` (deterministic, zero-spend), `OpenAIEmbedder` (network, gated),
+  `LocalEmbedder` (fastembed / sentence-transformers, on-device), and `CompositeEmbedder` (fuse several
+  embedders so a plain dot product equals their weighted fused cosine).
+- **Content-sha idempotency** — re-embed a row only when its text changes; a rerun over an unchanged corpus
+  embeds nothing, and changing one item's text re-embeds exactly that one.
+- **Typed errors** — `Unavailable(code=…)` and `NotSupported(capability, backend)` — never a silent degrade.
+- **Estimate-first** — `core.estimate(...)` sizes a job (tokens + `$`) with zero spend before you embed.
+- **Batch ingest** — `BatchIngest` for large corpora via a provider Batch API: chunked, resumable, a durable
+  handle ledger, every result accounted; realtime and non-blocking collect both supported.
+- **Optional seams, negotiated** — `capabilities()` + typed `NotSupported` mean expanding usage is additive:
+  a consumer starts on `search(collection, q, k)`, later adds `filters=…`, later calls `matrix()`, with no
+  rewrite.
+
+## Design
+
+veccore's line is **mechanism vs policy**: veccore ships the generic mechanisms + seams; each consumer builds
+its domain policy on top. The gate (spend/egress) and any observability sink are **injected**, so veccore has
+one hard dependency — `numpy` — and everything else (Qdrant, OpenAI, fastembed, sentence-transformers) is an
+optional extra you install only for the adapter you choose.
+
+## License
+
+MIT © 2026 Ash Damle. Developed in the [keelworks](https://github.com/ashdamle/keelworks) monorepo.

veccore-0.2.0/pyproject.toml

@@ -0,0 +1,66 @@
+[build-system]
+requires = ["setuptools>=77"]          # >=77 for PEP 639 SPDX license expression + license-files
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "veccore"
+version = "0.2.0"
+description = "Ports-and-adapters vector substrate: embed -> index -> search -> hydrate, backend/embedder-agnostic."
+readme = "README.md"
+requires-python = ">=3.11"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "Ash Damle" }]
+keywords = ["embeddings", "vector-search", "qdrant", "semantic-search", "ports-and-adapters", "retrieval"]
+# Core deps only. veccore ships MECHANISMS; heavy/optional backends and embedders are extras so a
+# consumer installs only what its chosen adapter needs. numpy is the sole hard dependency.
+dependencies = [
+    "numpy>=1.26",
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Software Development :: Libraries",
+    "Typing :: Typed",
+]
+
+[project.optional-dependencies]
+# Backends
+qdrant = ["qdrant-client>=1.9"]
+# Embedders (network / local)
+openai = ["openai>=1.30", "tiktoken>=0.7"]          # tiktoken → exact token counts for the paid estimate path
+st = ["sentence-transformers>=2.7"]
+fastembed = ["fastembed>=0.3"]                       # symgrep-codesearch[semantic] depends on veccore[fastembed]
+dev = [
+    "pytest>=8",
+    "pytest-timeout>=2.3",
+    "build>=1.2",
+    "twine>=5",
+]
+
+[project.urls]
+Homepage = "https://github.com/ashdamle/keelworks"
+Repository = "https://github.com/ashdamle/keelworks"
+Issues = "https://github.com/ashdamle/keelworks/issues"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+include = ["veccore*"]                               # ship ONLY veccore
+exclude = ["hookdispatch*", "tests*"]               # hookdispatch is a PARKED stub, never shipped in the veccore dist
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+# per-unit timeout so a hung embedder/network call fails the unit, not the suite
+# (feedback_chunk_never_single_shot: per-unit timeout)
+timeout = 120
+markers = [
+    "realspend: opt-in test that makes REAL metered OpenAI calls, capped ~$1 (deselected by default)",
+    "qdrant: requires a live Qdrant at :6333 (uses a throwaway veccore_test_* collection)",
+]
+addopts = "-m 'not realspend'"

veccore-0.2.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

veccore-0.2.0/src/veccore/__init__.py

@@ -0,0 +1,56 @@
+"""veccore — the ports-and-adapters vector substrate for the estate (keelworks L2).
+
+Generic MECHANISMS + clean SEAMS for `embed → index → search → hydrate`. Each consumer builds its domain
+POLICY on top; nothing consumer-specific lives here, and no consumer reinvents embed/index/idempotency.
+
+Quick start (a consumer's ENTIRE veccore-side adoption is: register a collection + hydrate from its own SoR):
+
+    from veccore import (VecCore, CollectionRegistry, CollectionSpec, AuditEnvelope,
+                         MockEmbedder, SqliteNumpyStore)
+
+    store = SqliteNumpyStore(path="/tmp/mine.sqlite")
+    core = VecCore(registry=CollectionRegistry(),
+                   stores={"sqlite_numpy": store},
+                   embedders={"mock": MockEmbedder()})
+    core.register(CollectionSpec(name="notes", backend="sqlite_numpy", embedder="mock", dim=64))
+
+    audit = AuditEnvelope(who="my_tool", why="index notes for reuse search")
+    core.embed_upsert("notes", [{"id": "n1", "text": "vector search over notes", "payload": {"src": "n1"}}], audit)
+
+    hits = core.search("notes", "nearest ideas", k=5, audit=AuditEnvelope(who="my_tool", why="find related"))
+    # each hit is {id, score, payload}; hydrate the full record from your own SoR by hit.id
+"""
+from .audit import AuditEnvelope, JsonlObserver, NullObserver, ObserverPort
+from .batch_ingest import BatchIngest, BatchReport
+from .core import Estimate, IngestResult, VecCore
+from .evaluate import bakeoff, recall_at_k, reciprocal_rank, score_queries
+from .embedders import CompositeEmbedder, LocalEmbedder, MockEmbedder, OpenAIEmbedder, make_embedder
+from .errors import (DimMismatch, GateRefused, ModelMismatch, NotSupported, OversizeRefused, Unavailable,
+                     VeccoreError)
+from .ports import EmbedderPort, GatePort, Hit, Item, NullGate, Point, VectorStorePort
+from .registry import CollectionRegistry, CollectionSpec
+from .adapters import QdrantStore, SqliteNumpyStore
+
+__version__ = "0.2.0"
+
+__all__ = [
+    # orchestrator + records
+    "VecCore", "IngestResult", "Estimate",
+    "CollectionRegistry", "CollectionSpec",
+    "AuditEnvelope", "ObserverPort", "NullObserver", "JsonlObserver",
+    "Item", "Point", "Hit",
+    # ports
+    "EmbedderPort", "VectorStorePort", "GatePort", "NullGate",
+    # embedders
+    "MockEmbedder", "OpenAIEmbedder", "LocalEmbedder", "CompositeEmbedder", "make_embedder",
+    # backends
+    "SqliteNumpyStore", "QdrantStore",
+    # batch seam
+    "BatchIngest", "BatchReport",
+    # measurement seam (choose/fuse embedders on evidence)
+    "bakeoff", "score_queries", "recall_at_k", "reciprocal_rank",
+    # errors
+    "VeccoreError", "Unavailable", "NotSupported", "DimMismatch", "ModelMismatch", "OversizeRefused",
+    "GateRefused",
+    "__version__",
+]

veccore-0.2.0/src/veccore/adapters/__init__.py

@@ -0,0 +1,6 @@
+"""Backend adapters: `sqlite_numpy` (in-process) and `qdrant` (server). Future faiss/pgvector slot in
+here by implementing VectorStorePort. Backend IDs are `sqlite_numpy` / `qdrant`, never `db`."""
+from .qdrant_store import QdrantStore
+from .sqlite_numpy_store import SqliteNumpyStore
+
+__all__ = ["SqliteNumpyStore", "QdrantStore"]

veccore-0.2.0/src/veccore/adapters/qdrant_store.py

@@ -0,0 +1,245 @@
+"""QdrantStore — the server backend (backend id "qdrant", never "db").
+
+For millions of points, an on-disk Qdrant collection with an ANN index is the right substrate (7thsense's
+9.3M-point corpus). SQLite/numpy cannot; a server can. This adapter is the mature 7thsense Qdrant path
+re-expressed on veccore's ports: dim-mismatch refusal, resumable existing-ids scroll, deterministic
+point-ids, and the OPTIONAL filter push-down seam.
+
+Capabilities: implements `search(filters=…)` (metadata / ACL push-down — permission enforcement in the
+query, O(index) not O(retrieved)) and is a Batch-ingest target. Does NOT implement `matrix()` — pulling
+millions of vectors client-side is not what a server backend is for; it raises typed NotSupported at that
+seam, exactly as the design intends (the honest negotiable "no").
+
+Point ids: Qdrant ids must be int or UUID, but a consumer's id is an arbitrary string. So the point id is
+`uuid5(NAMESPACE_URL, id)` (deterministic ⇒ re-upsert updates in place), and the real id + text_sha are
+carried in the payload under the reserved keys `_vid` / `_text_sha`, stripped from the payload returned to
+the consumer.
+
+Transport errors NEVER masquerade as 'missing'. `collection_exists()` separates a genuinely-absent
+collection from a timeout/auth failure: a real absence takes the create/empty path; any transport error
+PROPAGATES, so `ensure` never re-creates an existing collection and `existing_ids`/`stored_shas` never
+report points absent (which would re-embed the whole corpus, breaking the never-re-embed guarantee).
+"""
+from __future__ import annotations
+
+import uuid
+import warnings
+from typing import Any, Dict, List, Optional, Sequence, Tuple
+
+import numpy as np
+
+from ..errors import DimMismatch, ModelMismatch, NotSupported, Unavailable
+from ..ports import Hit, Point
+
+BACKEND = "qdrant"
+_VID = "_vid"            # reserved payload key: the consumer's real string id
+_TEXT_SHA = "_text_sha"  # reserved payload key: idempotency sha
+_MODEL = "_model"        # reserved payload key: the embedder identity (model guard; Qdrant has no
+#                          collection-metadata field, so the model is carried on every point and checked
+#                          via a 1-point read on ensure())
+
+
+def _point_uuid(vid: str) -> str:
+    """Deterministic Qdrant point id from the consumer's string id (uuid5 over a stable namespace) — so a
+    re-upsert of the same id updates in place rather than duplicating. Parsing an id, not a judgement."""
+    return str(uuid.uuid5(uuid.NAMESPACE_URL, vid))
+
+
+class QdrantStore:
+    backend = BACKEND
+
+    def __init__(self, url: str = "http://localhost:6333", timeout: int = 60, client=None):
+        self.url = url
+        self.timeout = timeout
+        self._client = client   # injectable for tests; else lazily constructed
+        # in-process cache of each collection's model (set by ensure(model=...)) so upsert can stamp
+        # `_model` on its points; the DURABLE record is the payload itself, re-read on a fresh process.
+        self._model_by_collection = {}
+        # count of foreign/legacy points (no reserved _vid) skipped by the last search — for a consumer
+        # that wants to observe a mixed/migration collection programmatically (also warned about).
+        self.last_search_foreign_skipped = 0
+
+    def _qc(self):
+        if self._client is None:
+            try:
+                from qdrant_client import QdrantClient
+            except ImportError as e:
+                raise Unavailable("qdrant-client is not installed — `pip install veccore[qdrant]`",
+                                  code="backend_unavailable") from e
+            # check_compatibility=False: server/client minor gap is fine for the stable REST ops used here
+            self._client = QdrantClient(url=self.url, timeout=self.timeout, check_compatibility=False)
+        return self._client
+
+    # ── core port ──────────────────────────────────────────────────────────────────────────────────────
+    def ensure(self, collection: str, dim: int, model: str = None) -> None:
+        from qdrant_client import models as qm
+        qc = self._qc()
+        # collection_exists() distinguishes "genuinely missing" from a transport/auth error: a timeout or
+        # 401 raises and PROPAGATES (fail-safe) rather than being mistaken for "missing" and triggering a
+        # create over an existing collection. Only a real absence takes the create path.
+        if not qc.collection_exists(collection):
+            qc.create_collection(
+                collection_name=collection,
+                vectors_config=qm.VectorParams(size=dim, distance=qm.Distance.COSINE, on_disk=True))
+            if model is not None:
+                self._model_by_collection[collection] = model
+            return
+        info = qc.get_collection(collection)
+        size = info.config.params.vectors.size
+        if size != dim:
+            raise DimMismatch(collection, size, dim)   # 7thsense guard, kept as a core invariant
+        if model is not None:
+            stored = self._first_model(qc, collection)   # None if the collection is empty / pre-guard
+            if stored is not None and stored != model:
+                raise ModelMismatch(collection, stored, model)
+            self._model_by_collection[collection] = model
+
+    def upsert(self, collection: str, points: Sequence[Point]) -> None:
+        from qdrant_client import models as qm
+        qc = self._qc()
+        model = self._model_by_collection.get(collection)
+        structs = []
+        for p in points:
+            payload = dict(p.payload)
+            payload[_VID] = p.id
+            payload[_TEXT_SHA] = p.text_sha
+            if model is not None:
+                payload[_MODEL] = model   # carries the model identity so ensure() can guard it later
+            structs.append(qm.PointStruct(
+                id=_point_uuid(p.id),
+                vector=np.asarray(p.vector, dtype=np.float32).ravel().tolist(),
+                payload=payload))
+        qc.upsert(collection_name=collection, wait=True, points=structs)
+
+    def search(self, collection: str, qvec: np.ndarray, k: int,
+               filters: Optional[Dict[str, Any]] = None) -> List[Hit]:
+        qc = self._qc()
+        flt = self._build_filter(filters) if filters else None
+        res = qc.query_points(
+            collection_name=collection,
+            query=np.asarray(qvec, dtype=np.float32).ravel().tolist(),
+            limit=k, query_filter=flt, with_payload=True).points
+        out = []
+        foreign = 0
+        for p in res:
+            payload = dict(p.payload or {})
+            vid = payload.pop(_VID, None)
+            payload.pop(_TEXT_SHA, None)
+            payload.pop(_MODEL, None)   # veccore-internal — never surfaced to the consumer
+            if vid is None:
+                # a legacy/foreign point (not written by veccore, so no reserved _vid). SKIP it rather than
+                # emit Hit(id=None) — a null id crashes a consumer doing int(hit.id); a mixed/migration
+                # collection must degrade safely, not poison the result set. Counted + warned, never silent.
+                foreign += 1
+                continue
+            out.append(Hit(id=vid, score=float(p.score), payload=payload))
+        self.last_search_foreign_skipped = foreign
+        if foreign:
+            warnings.warn(
+                f"veccore.QdrantStore.search: skipped {foreign} foreign point(s) with no {_VID} in "
+                f"collection {collection!r} (legacy/non-veccore points, e.g. a migration in progress) — "
+                f"not returned as null ids. Rebuild the collection through veccore to clear them.",
+                stacklevel=2)
+        return out
+
+    def delete(self, collection: str, ids: Sequence[str]) -> int:
+        """Remove points by id; return the count confirmed removed. Idempotent (absent ids → 0).
+
+        The DELETE is unconditional over the FULL requested set — Qdrant no-ops ids that are absent, so the
+        deletion always completes and can never leave a requested id half-deleted regardless of any lookup.
+        Qdrant's delete does not report a removed-count, so the returned number comes from a separate
+        pre-read (`retrieve`) of how many requested ids were present; that count is best-effort and only
+        ever CONSERVATIVE (a transient partial read under-reports, never over-reports) — the state change
+        does not depend on it, so a conservative count never corresponds to data left behind."""
+        from qdrant_client import models as qm
+        ids = list(ids)
+        if not ids:
+            return 0
+        qc = self._qc()
+        if not qc.collection_exists(collection):
+            return 0
+        puids = [_point_uuid(i) for i in ids]
+        confirmed = len(qc.retrieve(collection_name=collection, ids=puids,
+                                    with_payload=False, with_vectors=False))
+        qc.delete(collection_name=collection, wait=True,   # full requested set; absent ids are a no-op
+                  points_selector=qm.PointIdsList(points=puids))
+        return confirmed
+
+    def existing_ids(self, collection: str) -> set:
+        return set(self._scroll_field(collection, _VID).keys())
+
+    def stored_shas(self, collection: str) -> Dict[str, str]:
+        return self._scroll_field(collection, _TEXT_SHA)
+
+    def capabilities(self) -> Dict[str, bool]:
+        return {"matrix": False, "filters": True, "batch": True}
+
+    # ── optional capability seam (unimplemented here → honest typed NotSupported) ──────────────────────
+    def matrix(self, collection: str) -> Tuple[List[str], np.ndarray]:
+        raise NotSupported(
+            "matrix (materialize all vectors for X @ Xᵀ analytics)", self.backend,
+            "qdrant does not implement matrix() — pulling a whole collection client-side defeats the "
+            "server ANN index. Use the sqlite_numpy backend for in-process matrix analytics, or run the "
+            "analysis as server-side searches.")
+
+    # ── internals ────────────────────────────────────────────────────────────────────────────────────
+    def _first_model(self, qc, collection: str):
+        """The model identity carried by any one point (they all carry the same), or None if the
+        collection is empty / predates the guard. A transport error PROPAGATES (not caught) so a failed
+        read never silently skips the model guard."""
+        points, _off = qc.scroll(collection_name=collection, limit=1,
+                                 with_payload=[_MODEL], with_vectors=False)
+        for p in points:
+            return (p.payload or {}).get(_MODEL)
+        return None
+
+    def _scroll_field(self, collection: str, field: str) -> Dict[str, str]:
+        """Scroll the whole collection, payload-only, collecting {_vid: field_value}. This is what makes
+        an ingest RESUMABLE (a point already present is never re-embedded).
+
+        A genuinely-absent collection returns {} (nothing stored yet). A transport/auth error PROPAGATES
+        — it must NOT read as 'no points', because that would make an ingest treat every existing point as
+        new and re-embed the whole corpus (the never-re-embed guarantee broken, real money burned)."""
+        qc = self._qc()
+        if not qc.collection_exists(collection):
+            return {}
+        out: Dict[str, str] = {}
+        offset = None
+        while True:
+            points, offset = qc.scroll(
+                collection_name=collection, limit=4000, offset=offset,
+                with_payload=[_VID, field], with_vectors=False)
+            for p in points:
+                pl = p.payload or {}
+                vid = pl.get(_VID)
+                if vid is not None:
+                    out[vid] = pl.get(field)
+            if offset is None:
+                return out
+
+    def _build_filter(self, filters: Dict[str, Any]):
+        """Translate a generic filters dict into a Qdrant Filter — the metadata/ACL push-down MECHANISM;
+        the consumer builds its ACL POLICY on this (which principals/groups → which clauses).
+
+        Shapes:
+          {field: value}            → must match value (MatchValue)
+          {field: [v1, v2]}         → must match any of (MatchAny)
+          {"must": {...}, "should": {...}}  → explicit AND / OR groups (e.g. an ACL 'should' set)
+        A `should` group is deny-by-default OR: at least one clause must hold (7thsense's ACL parity).
+        """
+        from qdrant_client import models as qm
+
+        def conds(d: Dict[str, Any]):
+            out = []
+            for key, val in d.items():
+                if isinstance(val, (list, tuple, set)):
+                    out.append(qm.FieldCondition(key=key, match=qm.MatchAny(any=list(val))))
+                else:
+                    out.append(qm.FieldCondition(key=key, match=qm.MatchValue(value=val)))
+            return out
+
+        if "must" in filters or "should" in filters:
+            must = conds(filters.get("must", {})) or None
+            should = conds(filters.get("should", {})) or None
+            return qm.Filter(must=must, should=should)
+        return qm.Filter(must=conds(filters))