weft-embed 1.0.0__tar.gz

weft_embed-1.0.0/.gitignore

@@ -0,0 +1,78 @@
+# The donor checkout, reachable for reading and lifting only.
+# Never a build input: see CLAUDE.md and docs/README.md.
+/donor
+
+# Python
+__pycache__/
+*.py[cod]
+.venv/
+dist/
+build/
+*.egg-info/
+
+# Tooling
+.pytest_cache/
+.ruff_cache/
+.pyright/
+
+# `uv.lock` was here, filed between two caches. It is not a cache: `ci.yml` runs
+# `uv sync --frozen` in three of its four jobs, and every one of them died at that step
+# the first time CI ever ran. It is tracked now — the resolution CI installs and the
+# resolution a developer installs must be one artefact, which is the same argument
+# `docs/README.md` makes about single-sourcing anything two readers can disagree about.
+#
+# It constrains nobody downstream. A lockfile is not a dependency bound; what a consumer
+# resolves is decided by each distribution's `pyproject.toml`.
+
+# Session-local attempt counter for the guard_quality_gates.py PreToolUse hook — per-session
+# scratch, not a record anyone should read later. See the hook's module docstring.
+.claude/.gate-attempts.json
+
+# Secrets. `.env` holds live provider keys; only the documented, valueless
+# example is tracked. Listed before any tooling rule so a stray `git add -A`
+# cannot reach it.
+.env
+.env.*
+!.env.example
+
+# Corpus payload. Every subdirectory of /corpus is a materialised document set and
+# is deliberately untracked; `corpus/manifest.toml` and `scripts/fetch_corpus.py`
+# are the tracked artefact. `09` §4 V1 permits exactly this:
+# a corpus is "either redistributable or fetched by a pinned, checksummed script",
+# and the mRMR papers are published under publisher copyright, so committing them
+# would be redistribution this repository has no right to perform. The manifest
+# carries a sha256 per document, which is what makes the set reproducible without
+# the bytes being here. The pattern is a directory glob rather than a list of names
+# so that scaling the corpus up cannot silently start tracking a paper.
+/corpus/*/
+
+# Where a baseline run stages the corpus it indexes and writes the `weft.toml` it measures
+# through (`eval/run_baseline.py`). The staged copies are the same untracked papers one
+# directory over, and the configuration is reproduced by the harness rather than kept — what
+# is tracked is the run it produced, under `eval/baselines/`.
+/.baseline-run/
+
+# Working artefacts of a build session — a generated map of the codebase and a design
+# record produced while planning. Untracked on purpose: `docs/README.md` routes every
+# document this project owns, and a design that matters belongs in the `docs/` file
+# that owns its content, not in a root-level file nothing points at.
+/.phase2-*.md
+/.phase3-*.md
+/.phase4-*.md
+
+# Gate-session preparation: the Bring lists of `docs/05-grilling-sessions.md`, measured on the day
+# a session is about to run. Untracked for the same reason — the session's outcome belongs in
+# `docs/README.md`'s decision log and in the reference document the decision changes, never here.
+/.gate-brief-*.md
+
+# Claude Code local state
+.claude/settings.local.json
+
+.DS_Store
+
+# The build harness driving Phase 2, alongside its brief, findings and design record.
+/.phase2-build.js
+
+# Transient: harvested subagent findings, promoted into docs/lessons.md by implement-ll.
+# Never committed — its content belongs in the queue or nowhere.
+.claude/lessons-spool.md

weft_embed-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Adam Krysztopa
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

weft_embed-1.0.0/NOTICE

@@ -0,0 +1,24 @@
+Weft
+Copyright (c) 2026 Adam Krysztopa
+
+This product is licensed under the MIT License. See the LICENSE file at the root
+of this repository.
+
+--------------------------------------------------------------------------------
+Original work
+--------------------------------------------------------------------------------
+
+**Weft contains no source text from any other codebase.** Every line here is
+written for this project, against this project's contracts.
+
+This is a rule, not a description of the current state: no file may be copied or
+adapted from another project's source, and no third-party source text may be
+pasted into this repository. Where a prior system informed a design, what was
+carried across is understanding — an approach, an ordering, a measurement, a
+reason a guard exists — restated in this project's own words and implemented
+fresh. Copyright does not reach any of that, and nothing in this repository
+depends on a licence granted by anyone else.
+
+`docs/04-donor-inventory.md` records what was learned from prior work and what
+was deliberately not taken. It is a design record. Nothing in it authorises a
+copy, because copying is not permitted here at all.

weft_embed-1.0.0/PKG-INFO

@@ -0,0 +1,9 @@
+Metadata-Version: 2.5
+Name: weft-embed
+Version: 1.0.0
+Summary: First-party embedding pack. Publishes the Embedder contract.
+License-Expression: MIT
+License-File: LICENSE
+License-File: NOTICE
+Requires-Python: >=3.12
+Requires-Dist: weft-kernel<1.0.0,>=0.1.0

weft_embed-1.0.0/pyproject.toml

@@ -0,0 +1,16 @@
+[project]
+name = "weft-embed"
+version = "1.0.0"
+description = "First-party embedding pack. Publishes the Embedder contract."
+requires-python = ">=3.12"
+license = "MIT"
+license-files = ["LICENSE", "NOTICE"]
+dependencies = ["weft-kernel>=0.1.0,<1.0.0"]
+
+# The one entry point a pack declares (`docs/02-extension-model.md` section 2).
+[project.entry-points."weft.packs"]
+embed = "weft_embed:register"
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"

weft_embed-1.0.0/src/weft_embed/__init__.py

@@ -0,0 +1,41 @@
+"""First-party embedding pack.
+
+Publishes the `Embedder` contract, in `contract.py`, and registers
+`HashEmbedder` (`hash_embedder.py`) — a deterministic local embedder that is
+**not a quality component**, per `docs/06-phase-0-build.md` step 8. This
+pack exists at all because of a forced conclusion, not a design choice: G4
+forbids a `Store` from embedding, G2 has not placed the embed step, and a
+walking skeleton must not need a model download or an API key to prove that
+indexing produces stored nodes.
+
+Registered through the same public `weft.packs` entry point any third-party
+pack uses — fitness function 2 — with no shortcut for being first-party.
+"""
+
+from pydantic import BaseModel, ConfigDict
+
+from weft_embed.contract import EMBEDDER_CONTRACT_VERSION, Embedder
+from weft_embed.hash_embedder import HashEmbedder, HashEmbedderConfig
+from weft_kernel.discovery import PackRegistrar
+
+
+class Settings(BaseModel):
+    """`weft-embed` takes no pack settings — an empty model is still the required shape."""
+
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+
+def register(registrar: PackRegistrar, settings: Settings) -> None:
+    """Register `HashEmbedder` as `"hash"` for `Embedder`. The only plugin this pack ships."""
+    del settings
+    registrar.add(Embedder, "hash", HashEmbedder)
+
+
+__all__ = [
+    "EMBEDDER_CONTRACT_VERSION",
+    "Embedder",
+    "HashEmbedder",
+    "HashEmbedderConfig",
+    "Settings",
+    "register",
+]

weft_embed-1.0.0/src/weft_embed/contract.py

@@ -0,0 +1,66 @@
+"""The `Embedder` contract — published here, never by the kernel.
+
+Specified in `docs/06-phase-0-build.md` step 8. `weft-embed` is the fourth
+pack Phase 0 needs and the plan did not originally anticipate — the reasoning
+is forced, not a design flourish: G4 forbids a `Store` from embedding
+(`docs/02-extension-model.md` → *The store contract family*, "stores never
+embed"), G2 has not decided whether embedding is a pipeline stage or a step
+inside another one, and a walking skeleton must not depend on a model
+download or an API key. Phase 0 puts embedding in the pipeline list as its
+own stage — `06`'s minimal, reversible choice for G2's first trap, not an
+answer to G2 — which is exactly what forces this contract to exist somewhere
+a `Store` cannot own it.
+
+**`Embedder` needs no boundary type of its own**, for the same reason
+`weft_chunk.contract.Chunker` does not: `docs/02-extension-model.md` →
+*Composition is typed and checked at load* states the ingest path is
+`Stage[Seq[Node], Seq[Node]]` throughout, and embedding a `Node` that already
+has content but no vector yet is squarely inside that stretch — a `Node` in,
+the same `Node` with `embedding` set (via `Node.with_embedding`) out.
+
+**`Embedder` declares `Stage[Sequence[Node], Sequence[Node]]` as one of its
+own bases**, per `weft_kernel.runner`'s documented convention — the linear
+runner reads a pipeline stage's `In`/`Out` off the *contract* named in its
+`StageSpec`, via `__orig_bases__`, never off the plugin implementing it.
+`@runtime_checkable` makes capability checkable by `isinstance` rather than
+by a declared flag, the same property every other Phase 0 contract shares.
+`Embedder.__protocol_attrs__` is exactly `{'run'}` — `Stage` itself declares
+nothing beyond `run` (see `weft_kernel.runner`'s module docstring), so a
+plugin implementing only `run` satisfies `Embedder`, full stop.
+
+`version` is readable off the class (`Embedder.version`) but is not part of
+that `isinstance` membership — see `weft_extract.contract`'s module
+docstring for the full reasoning behind the `if TYPE_CHECKING:` /
+assign-after-the-class-body split, which applies here unchanged.
+"""
+
+from collections.abc import Sequence
+from typing import TYPE_CHECKING, ClassVar, Protocol, runtime_checkable
+
+from weft_kernel.context import Context
+from weft_kernel.payload import Node, Outcome
+from weft_kernel.runner import Stage
+
+#: Fitness function 6's subject for this contract — see the module docstring.
+EMBEDDER_CONTRACT_VERSION = "1.0.0"
+
+
+@runtime_checkable
+class Embedder(Stage[Sequence[Node], Sequence[Node]], Protocol):
+    """Attaches an embedding to each `Node` it is handed.
+
+    One method, domain types on both sides, exactly `Chunker`'s shape one
+    stage later in the pipeline. An embedder that finds nothing to embed
+    (an empty batch) answers `NothingToProduce`, not an empty `Produced([])`
+    — the same donor-trap fix every other Phase 0 contract documents.
+    """
+
+    if TYPE_CHECKING:
+        #: See the module docstring — declared only for a type checker, assigned for real
+        #: after the class body, so it never joins `__protocol_attrs__`.
+        version: ClassVar[str]
+
+    async def run(self, payload: Sequence[Node], ctx: Context) -> Outcome[Sequence[Node]]: ...
+
+
+Embedder.version = EMBEDDER_CONTRACT_VERSION

weft_embed-1.0.0/src/weft_embed/hash_embedder.py

@@ -0,0 +1,100 @@
+"""`HashEmbedder` — a deterministic, content-hashed vector. **Not a quality embedder.**
+
+Specified in `docs/06-phase-0-build.md` step 8: "Phase 0 ships a deterministic
+local embedder (hashing to a fixed-dimension vector) whose only job is to be
+a real vector produced by a real stage. It is not a quality component and
+its docstring should say so." This is that docstring, said plainly: `HashEmbedder`
+carries no semantic understanding of content whatsoever. Two documents about
+unrelated topics that happen to hash to nearby components are not "similar"
+in any sense a retrieval strategy should trust — this class exists only so
+that Phase 0's ingest pipeline has a real `Vector`, produced by a real stage,
+flowing into a real `NodeStore.search_vector` call, with no model download
+and no API key standing between a clean checkout and a passing test. Replace
+it with a real embedding model before trusting a single retrieval result
+against it.
+
+**Deterministic by construction, not by convention.** Every component is a
+SHA-256 digest of the node's content concatenated with that component's own
+index, so the same content always hashes to the same vector — required for
+`weft index`'s re-index-is-idempotent property (`docs/02-extension-model.md`
+→ *Identity is a content-addressed digest*) to hold all the way through
+storage, not just at the `Node` layer. It is not, and must never become,
+a hash-based *approximation* of semantic similarity: nothing here reads
+tokens, n-grams or any structure in `content` at all.
+
+**Configuration is `dimension`, and nothing else.** No model name, no API
+endpoint, no batching knob — there is no model to name, and pretending
+otherwise (a `model: "text-embedding-3-small"` field that is silently
+ignored) would be exactly the kind of plausible-but-false configuration
+surface the project's catch-specific-exceptions rule already refuses at the
+error-handling layer. `dimension` is real: it is the length of the vector
+this stage actually produces.
+"""
+
+import hashlib
+from collections.abc import Sequence
+
+from pydantic import BaseModel, ConfigDict, model_validator
+
+from weft_kernel.context import Context
+from weft_kernel.payload import Node, NothingToProduce, Outcome, Produced, Vector
+
+#: Matches `weft-store`'s pgvector column, which is declared without a fixed
+#: dimension — see `docs/06-phase-0-build.md` step 8 and `weft_store.pgvector_store`.
+#: Any positive dimension works; this is only the default a `with:` block may omit.
+_DEFAULT_DIMENSION = 64
+
+#: The width of the slice of each SHA-256 digest this module turns into one component.
+_UINT64_RANGE = 2**64
+
+
+class HashEmbedderConfig(BaseModel):
+    """`HashEmbedder`'s `with:` config — one real field. See the module docstring for why."""
+
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    dimension: int = _DEFAULT_DIMENSION
+
+    @model_validator(mode="after")
+    def _dimension_is_positive(self) -> "HashEmbedderConfig":
+        if self.dimension < 1:
+            raise ValueError(
+                f"dimension must be at least 1 — a Vector cannot be empty (got {self.dimension})"
+            )
+        return self
+
+
+class HashEmbedder:
+    """Attaches a deterministic, content-hashed `Vector` to every node it is handed.
+
+    **Not a quality component** — see the module docstring. Satisfies
+    `weft_embed.contract.Embedder` structurally: this class never imports it,
+    the same path every third-party embedder pack is expected to take.
+    """
+
+    def __init__(self, config: HashEmbedderConfig | None = None) -> None:
+        self._config = config if config is not None else HashEmbedderConfig()
+
+    async def run(self, payload: Sequence[Node], ctx: Context) -> Outcome[Sequence[Node]]:
+        del ctx  # no service or locale this stage needs
+        if not payload:
+            return NothingToProduce(reason="no nodes to embed")
+        embedded = [
+            node.with_embedding(_hash_vector(node.content, self._config.dimension))
+            for node in payload
+        ]
+        return Produced(value=embedded)
+
+
+def _hash_vector(content: str, dimension: int) -> Vector:
+    """`dimension` components, each a deterministic function of `content` and its own index."""
+    encoded = content.encode("utf-8")
+    components = tuple(_component(encoded, index) for index in range(dimension))
+    return Vector(values=components)
+
+
+def _component(encoded: bytes, index: int) -> float:
+    """One component: a SHA-256 digest of `encoded` and `index`, mapped onto `[-1.0, 1.0)`."""
+    digest = hashlib.sha256(encoded + index.to_bytes(4, byteorder="big")).digest()
+    raw = int.from_bytes(digest[:8], byteorder="big")
+    return (raw / _UINT64_RANGE) * 2.0 - 1.0