hermes-memory-pgvector 0.5.0__tar.gz → 0.5.2__tar.gz

{hermes_memory_pgvector-0.5.0 → hermes_memory_pgvector-0.5.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hermes-memory-pgvector
-Version: 0.5.0
+Version: 0.5.2
 Summary: Postgres + pgvector memory provider plugin for hermes-agent. Multi-agent storage layer with per-minion themes, identity governance, agent attribution + delegation provenance, async writer, no LLM in the memory hot path.
 Author: Andrea Borghi
 License: BSD-3-Clause
@@ -189,6 +189,28 @@ Correctness release from a full-codebase review. **No schema changes and no new
 - **`save_config` stops deleting your settings.** It replaced the whole `plugins.pgvector` block with schema-declared keys, silently dropping hand-edited ones that are read at runtime (`identity_aliases`, `embed_write_backoff`). It merges now.
 - **Fail-soft hardening** (invariant #4): `sync_turn` is wrapped, config casts are guarded, and the recall tools coerce non-string `query`/`scope`/`target` instead of raising `AttributeError` out of the hook. `hermes-pgvector install --remove` now fails closed and requires `--force` on a directory that isn't a generated shim, instead of deleting it outright.
 
+## New in v0.5.1 — `memory remove` no longer wipes a whole theme
+
+Patch release, but **upgrade promptly**: it fixes a data-loss bug. No schema changes, no migrations, no API changes.
+
+- **`remove` deleted every mirrored entry for a theme, not one.** `_worker` passed `old_text=item.content` — but the built-in tool's remove op carries its target in `old_text` and leaves `content` empty, and the host forwards `old_text` via *metadata*. So `item.content` was always `""`, `store.remove` built `content LIKE '%%'`, and that matches every row: a single `memory remove` deleted the entire mirror for that `(agent_identity, target)`. Verified against Postgres — `DELETE … WHERE c LIKE '%%'` removes all rows. `_worker` now reads `extra["old_text"]`, and `store.remove()` **refuses an empty pattern outright**, so no caller can reach that delete by omission. `remove()` also now deletes **at most one row** (lowest id), matching both the built-in tool — which requires a unique match and errors on ambiguity — and this class's own `replace()`. (The built-in store was never affected; only the pgvector mirror. No loss occurred on the reference deployment: every theme's history is continuous.)
+
+Found in production: one `memory_entries` row sat with a NULL embedding and **zero-length content**, arrived through the built-in tool's `replace` path. Two defects met there.
+
+- **Nothing rejected empty content on write.** `on_memory_write` filtered on `target` and `action` but never on content, so an `add`/`replace` carrying nothing created a row that can never be embedded — `embed()` raises `EmbeddingError("empty input")` unconditionally for empty or whitespace text. Such writes are now ignored (`remove` is exempt: it legitimately arrives with empty content and targets the row via `old_text`).
+- **`backfill_null_embeddings` retried it forever.** The sweep selected `WHERE embedding IS NULL` with no content filter, so every nightly run re-fetched the row, called `embed()`, failed, and moved on — permanently pinning `failed` above zero and making `remaining == 0` unreachable. That is the damaging half: it destroys the one signal an operator watches, because you can no longer distinguish a permanently-stuck row from a new genuine failure. Un-embeddable rows are now **skipped and reported separately** as `unembeddable` (skipping them silently would be equally misleading), so `remaining` can actually reach zero again.
+
+## New in v0.5.2 — documentation only
+
+**No code changes.** `git diff v0.5.1..v0.5.2` touches only `README.md` and one test file; nothing under `hermes_pgvector/` differs, so the installed behaviour is byte-for-byte identical to v0.5.1. There is no reason to redeploy for this release.
+
+It exists because PyPI renders a project's README **frozen at upload time**: two fixes that landed after v0.5.1 shipped were visible on GitHub but not on the package page.
+
+- **The v0.5.1 release notes were out of order.** The section sat before v0.5.0 instead of after it, so the newest release was buried mid-list. These sections run oldest-to-newest.
+- A test asserted a `failed` count against a dry-run baseline that is hardcoded to `0`, making the comparison a no-op. Asserted directly now, with the reasoning recorded rather than the misleading framing.
+
+If you are on v0.5.1 you already have every fix in this release. If you are on **v0.5.0 or earlier, upgrade** — v0.5.1 fixed a data-loss bug where a single `memory remove` deleted a whole theme's mirrored memory.
+
 ## Multi-agent / per-minion themes
 
 Each systemd-run minion sets one header on its OpenAI client; everything else flows automatically:

{hermes_memory_pgvector-0.5.0 → hermes_memory_pgvector-0.5.2}/README.md

@@ -156,6 +156,28 @@ Correctness release from a full-codebase review. **No schema changes and no new
 - **`save_config` stops deleting your settings.** It replaced the whole `plugins.pgvector` block with schema-declared keys, silently dropping hand-edited ones that are read at runtime (`identity_aliases`, `embed_write_backoff`). It merges now.
 - **Fail-soft hardening** (invariant #4): `sync_turn` is wrapped, config casts are guarded, and the recall tools coerce non-string `query`/`scope`/`target` instead of raising `AttributeError` out of the hook. `hermes-pgvector install --remove` now fails closed and requires `--force` on a directory that isn't a generated shim, instead of deleting it outright.
 
+## New in v0.5.1 — `memory remove` no longer wipes a whole theme
+
+Patch release, but **upgrade promptly**: it fixes a data-loss bug. No schema changes, no migrations, no API changes.
+
+- **`remove` deleted every mirrored entry for a theme, not one.** `_worker` passed `old_text=item.content` — but the built-in tool's remove op carries its target in `old_text` and leaves `content` empty, and the host forwards `old_text` via *metadata*. So `item.content` was always `""`, `store.remove` built `content LIKE '%%'`, and that matches every row: a single `memory remove` deleted the entire mirror for that `(agent_identity, target)`. Verified against Postgres — `DELETE … WHERE c LIKE '%%'` removes all rows. `_worker` now reads `extra["old_text"]`, and `store.remove()` **refuses an empty pattern outright**, so no caller can reach that delete by omission. `remove()` also now deletes **at most one row** (lowest id), matching both the built-in tool — which requires a unique match and errors on ambiguity — and this class's own `replace()`. (The built-in store was never affected; only the pgvector mirror. No loss occurred on the reference deployment: every theme's history is continuous.)
+
+Found in production: one `memory_entries` row sat with a NULL embedding and **zero-length content**, arrived through the built-in tool's `replace` path. Two defects met there.
+
+- **Nothing rejected empty content on write.** `on_memory_write` filtered on `target` and `action` but never on content, so an `add`/`replace` carrying nothing created a row that can never be embedded — `embed()` raises `EmbeddingError("empty input")` unconditionally for empty or whitespace text. Such writes are now ignored (`remove` is exempt: it legitimately arrives with empty content and targets the row via `old_text`).
+- **`backfill_null_embeddings` retried it forever.** The sweep selected `WHERE embedding IS NULL` with no content filter, so every nightly run re-fetched the row, called `embed()`, failed, and moved on — permanently pinning `failed` above zero and making `remaining == 0` unreachable. That is the damaging half: it destroys the one signal an operator watches, because you can no longer distinguish a permanently-stuck row from a new genuine failure. Un-embeddable rows are now **skipped and reported separately** as `unembeddable` (skipping them silently would be equally misleading), so `remaining` can actually reach zero again.
+
+## New in v0.5.2 — documentation only
+
+**No code changes.** `git diff v0.5.1..v0.5.2` touches only `README.md` and one test file; nothing under `hermes_pgvector/` differs, so the installed behaviour is byte-for-byte identical to v0.5.1. There is no reason to redeploy for this release.
+
+It exists because PyPI renders a project's README **frozen at upload time**: two fixes that landed after v0.5.1 shipped were visible on GitHub but not on the package page.
+
+- **The v0.5.1 release notes were out of order.** The section sat before v0.5.0 instead of after it, so the newest release was buried mid-list. These sections run oldest-to-newest.
+- A test asserted a `failed` count against a dry-run baseline that is hardcoded to `0`, making the comparison a no-op. Asserted directly now, with the reasoning recorded rather than the misleading framing.
+
+If you are on v0.5.1 you already have every fix in this release. If you are on **v0.5.0 or earlier, upgrade** — v0.5.1 fixed a data-loss bug where a single `memory remove` deleted a whole theme's mirrored memory.
+
 ## Multi-agent / per-minion themes
 
 Each systemd-run minion sets one header on its OpenAI client; everything else flows automatically:

{hermes_memory_pgvector-0.5.0 → hermes_memory_pgvector-0.5.2}/hermes_memory_pgvector.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hermes-memory-pgvector
-Version: 0.5.0
+Version: 0.5.2
 Summary: Postgres + pgvector memory provider plugin for hermes-agent. Multi-agent storage layer with per-minion themes, identity governance, agent attribution + delegation provenance, async writer, no LLM in the memory hot path.
 Author: Andrea Borghi
 License: BSD-3-Clause
@@ -189,6 +189,28 @@ Correctness release from a full-codebase review. **No schema changes and no new
 - **`save_config` stops deleting your settings.** It replaced the whole `plugins.pgvector` block with schema-declared keys, silently dropping hand-edited ones that are read at runtime (`identity_aliases`, `embed_write_backoff`). It merges now.
 - **Fail-soft hardening** (invariant #4): `sync_turn` is wrapped, config casts are guarded, and the recall tools coerce non-string `query`/`scope`/`target` instead of raising `AttributeError` out of the hook. `hermes-pgvector install --remove` now fails closed and requires `--force` on a directory that isn't a generated shim, instead of deleting it outright.
 
+## New in v0.5.1 — `memory remove` no longer wipes a whole theme
+
+Patch release, but **upgrade promptly**: it fixes a data-loss bug. No schema changes, no migrations, no API changes.
+
+- **`remove` deleted every mirrored entry for a theme, not one.** `_worker` passed `old_text=item.content` — but the built-in tool's remove op carries its target in `old_text` and leaves `content` empty, and the host forwards `old_text` via *metadata*. So `item.content` was always `""`, `store.remove` built `content LIKE '%%'`, and that matches every row: a single `memory remove` deleted the entire mirror for that `(agent_identity, target)`. Verified against Postgres — `DELETE … WHERE c LIKE '%%'` removes all rows. `_worker` now reads `extra["old_text"]`, and `store.remove()` **refuses an empty pattern outright**, so no caller can reach that delete by omission. `remove()` also now deletes **at most one row** (lowest id), matching both the built-in tool — which requires a unique match and errors on ambiguity — and this class's own `replace()`. (The built-in store was never affected; only the pgvector mirror. No loss occurred on the reference deployment: every theme's history is continuous.)
+
+Found in production: one `memory_entries` row sat with a NULL embedding and **zero-length content**, arrived through the built-in tool's `replace` path. Two defects met there.
+
+- **Nothing rejected empty content on write.** `on_memory_write` filtered on `target` and `action` but never on content, so an `add`/`replace` carrying nothing created a row that can never be embedded — `embed()` raises `EmbeddingError("empty input")` unconditionally for empty or whitespace text. Such writes are now ignored (`remove` is exempt: it legitimately arrives with empty content and targets the row via `old_text`).
+- **`backfill_null_embeddings` retried it forever.** The sweep selected `WHERE embedding IS NULL` with no content filter, so every nightly run re-fetched the row, called `embed()`, failed, and moved on — permanently pinning `failed` above zero and making `remaining == 0` unreachable. That is the damaging half: it destroys the one signal an operator watches, because you can no longer distinguish a permanently-stuck row from a new genuine failure. Un-embeddable rows are now **skipped and reported separately** as `unembeddable` (skipping them silently would be equally misleading), so `remaining` can actually reach zero again.
+
+## New in v0.5.2 — documentation only
+
+**No code changes.** `git diff v0.5.1..v0.5.2` touches only `README.md` and one test file; nothing under `hermes_pgvector/` differs, so the installed behaviour is byte-for-byte identical to v0.5.1. There is no reason to redeploy for this release.
+
+It exists because PyPI renders a project's README **frozen at upload time**: two fixes that landed after v0.5.1 shipped were visible on GitHub but not on the package page.
+
+- **The v0.5.1 release notes were out of order.** The section sat before v0.5.0 instead of after it, so the newest release was buried mid-list. These sections run oldest-to-newest.
+- A test asserted a `failed` count against a dry-run baseline that is hardcoded to `0`, making the comparison a no-op. Asserted directly now, with the reasoning recorded rather than the misleading framing.
+
+If you are on v0.5.1 you already have every fix in this release. If you are on **v0.5.0 or earlier, upgrade** — v0.5.1 fixed a data-loss bug where a single `memory remove` deleted a whole theme's mirrored memory.
+
 ## Multi-agent / per-minion themes
 
 Each systemd-run minion sets one header on its OpenAI client; everything else flows automatically:

{hermes_memory_pgvector-0.5.0 → hermes_memory_pgvector-0.5.2}/hermes_memory_pgvector.egg-info/SOURCES.txt

@@ -21,6 +21,7 @@ hermes_pgvector/migrations/004_runtime_grants.sql
 tests/test_async_writer.py
 tests/test_config_coercion.py
 tests/test_embed_timeouts.py
+tests/test_empty_content.py
 tests/test_exclude_identities_live.py
 tests/test_hybrid_search.py
 tests/test_identity.py

{hermes_memory_pgvector-0.5.0 → hermes_memory_pgvector-0.5.2}/hermes_pgvector/__init__.py

@@ -888,6 +888,27 @@ class PgvectorMemoryProvider(MemoryProvider):
         if action not in ("add", "replace", "remove"):
             logger.debug("pgvector ignoring unknown action: %r", action)
             return
+        # An add/replace with no content produces a row that can NEVER be
+        # embedded: embed() raises EmbeddingError("empty input") on empty or
+        # whitespace text. Such a row is retried by every nightly backfill
+        # forever, always fails, and permanently prevents the NULL-embedding
+        # count from reaching zero -- destroying the one signal an operator
+        # watches. It also carries no information worth mirroring. `remove`
+        # is exempt: it legitimately arrives with empty content and targets
+        # the row via old_text.
+        if action in ("add", "replace") and not (content or "").strip():
+            # Not routine. on_memory_write only fires for writes the built-in
+            # tool COMMITTED, and it rejects empty add/replace content -- so
+            # arriving here means real content landed on disk and reached us
+            # as ''. Skipping keeps an un-embeddable row out of the table, but
+            # the mirror is now missing an entry the agent believes it saved,
+            # which is worth more than a debug line.
+            logger.warning(
+                "pgvector: %r for target=%r arrived with empty content; "
+                "skipping (the built-in store has it, the mirror will not)",
+                action, target,
+            )
+            return
 
         meta = dict(metadata or {})
         meta.setdefault("session_id", self._session_id)
@@ -957,11 +978,27 @@ class PgvectorMemoryProvider(MemoryProvider):
                         metadata=item.metadata,
                     )
             elif item.action == "remove":
-                self._store.remove(
-                    agent_identity=item.agent_identity,
-                    target=item.target,
-                    old_text=item.content,
-                )
+                # The removal target lives in extra["old_text"], NOT in content.
+                # The built-in tool's remove op takes old_text and leaves content
+                # empty (tools/memory_tool.py: remove -> store.remove(target,
+                # old_text)), and the host forwards old_text via METADATA
+                # (memory_manager.notify_memory_tool_write). Reading item.content
+                # here meant remove() was called with "", which becomes
+                # `content LIKE '%%'` -- matching every row and deleting the
+                # entire mirror for that (agent_identity, target).
+                old_text = item.extra.get("old_text") or item.content
+                if not (old_text or "").strip():
+                    logger.warning(
+                        "pgvector refusing remove with no old_text for %s/%s "
+                        "(would match every row)",
+                        item.agent_identity, item.target,
+                    )
+                else:
+                    self._store.remove(
+                        agent_identity=item.agent_identity,
+                        target=item.target,
+                        old_text=old_text,
+                    )
             elif item.action == "turn":
                 role = item.extra.get("role") or "user"
                 sid = item.extra.get("session_id") or "default"

{hermes_memory_pgvector-0.5.0 → hermes_memory_pgvector-0.5.2}/hermes_pgvector/__main__.py

@@ -277,10 +277,17 @@ def cmd_stats(args) -> int:
     conv = store.count_turns()
     print(f"memory_entries: {mem} rows")
     print(f"conversations:  {conv} rows")
-    # null-embedding counts (dry-run backfill returns remaining-null per table)
+    # null-embedding counts (dry-run backfill returns remaining-null per table).
+    # `remaining` counts only rows that CAN be embedded; empty/whitespace rows
+    # are reported separately, because they never shrink and would otherwise
+    # make this number look permanently stuck for no actionable reason.
     nulls = store.backfill_null_embeddings(embed_fn=lambda t: [0.0] * 768, dry_run=True)
     for table, info in nulls.items():
-        print(f"  {table}: {info['remaining']} null-embedding rows")
+        line = f"  {table}: {info['remaining']} null-embedding rows (backfillable)"
+        stuck = info.get("unembeddable")
+        if stuck:
+            line += f", {stuck} un-embeddable (empty content — will never backfill)"
+        print(line)
     if store.ensure_migration_002_applied():
         print("migration 002: applied — per-agent attribution:")
         for row in store.agent_attribution():

{hermes_memory_pgvector-0.5.0 → hermes_memory_pgvector-0.5.2}/hermes_pgvector/plugin.yaml

@@ -1,5 +1,5 @@
 name: pgvector
-version: 0.5.0
+version: 0.5.2
 description: "Postgres + pgvector storage layer for hermes-agent. Mirrors built-in MEMORY.md / USER.md + captures every substantive (user, assistant) chat turn into a conversations table. Per-agent themes via X-Hermes-Session-Key with v0.4 identity governance (PII/DM-key bucketing, bench isolation, allow-list). Agent attribution + parent->child delegation provenance (memory_agents / memory_agent_edges). 768-dim semantic recall with hybrid vector+full-text (RRF) ranking, embedding backfill, conversation TTL. Async writer + connection pool. No LLM mediation."
 pip_dependencies:
   - "psycopg[binary]>=3.3.5,<4"

{hermes_memory_pgvector-0.5.0 → hermes_memory_pgvector-0.5.2}/hermes_pgvector/store.py

@@ -296,18 +296,41 @@ class MemoryStore:
         target: str,
         old_text: str,
     ) -> int:
-        """Delete entries in (agent_identity, target) matching old_text substring.
-
-        Returns the number of rows deleted.
+        """Delete THE entry in (agent_identity, target) matching old_text.
+
+        Deletes at most ONE row (lowest id), matching both the built-in tool
+        and this class's own replace(). The built-in requires a UNIQUE match
+        and errors on ambiguity (memory_tool_store._edit -> _find_unique_match),
+        so one built-in remove is one entry; a mirror that deleted every
+        substring match would remove strictly more -- and the mirror is the
+        side that accumulates stale rows MEMORY.md no longer has, so "every
+        match" is wider here than it would be there.
+
+        Returns the number of rows deleted (0 or 1).
+
+        REFUSES an empty or whitespace-only old_text. `%{""}%` is `LIKE '%%'`,
+        which matches every row -- so a caller that lost the removal target
+        would silently delete the entire mirror for that (agent_identity,
+        target) instead of one entry. A delete this destructive must never be
+        reachable by omission; the caller has to say what it means to remove.
         """
+        if not (old_text or "").strip():
+            raise ValueError(
+                "remove() requires a non-empty old_text: an empty pattern is "
+                "LIKE '%%', which would delete every entry in this scope"
+            )
         with self._get_pool().connection() as conn:
             with conn.cursor() as cur:
                 cur.execute(
                     """
                     DELETE FROM memory_entries
-                     WHERE agent_identity = %s
-                       AND target = %s
-                       AND content LIKE %s
+                     WHERE id = (
+                         SELECT id FROM memory_entries
+                          WHERE agent_identity = %s
+                            AND target = %s
+                            AND content LIKE %s
+                          ORDER BY id LIMIT 1
+                     )
                     """,
                     (agent_identity, target, f"%{_escape_like(old_text)}%"),
                 )
@@ -922,7 +945,15 @@ class MemoryStore:
         fail fast, never write a wrong-dim vector). If the embed endpoint is
         unreachable, the run aborts cleanly (nothing to backfill right now).
 
-        Returns {table: {processed, succeeded, failed, remaining}}.
+        Rows whose content is empty or whitespace are EXCLUDED, not failed.
+        embed() raises EmbeddingError("empty input") on such text, so selecting
+        them means retrying a guaranteed failure on every nightly run forever,
+        permanently pinning `failed` above zero and making "remaining == 0"
+        unreachable -- which destroys the one signal an operator watches. They
+        are reported separately as `unembeddable` so they are skipped, not
+        hidden.
+
+        Returns {table: {processed, succeeded, failed, remaining, unembeddable}}.
         """
         tables = self._assert_whitelisted(tables)
         result: Dict[str, Dict[str, Any]] = {}
@@ -933,7 +964,8 @@ class MemoryStore:
             except Exception as exc:  # noqa: BLE001
                 logger.warning("backfill aborted — embed endpoint unavailable: %s", str(exc)[:200])
                 return {t: {"processed": 0, "succeeded": 0, "failed": 0,
-                            "remaining": None, "note": "embed-unavailable"} for t in tables}
+                            "remaining": None, "unembeddable": None,
+                            "note": "embed-unavailable"} for t in tables}
             if not isinstance(probe, list) or len(probe) != 768:
                 got = len(probe) if isinstance(probe, list) else type(probe).__name__
                 raise ValueError(
@@ -944,10 +976,31 @@ class MemoryStore:
         for t in tables:
             with self._get_pool().connection() as conn:
                 with conn.cursor() as cur:
-                    cur.execute(f"SELECT count(*) FROM {t} WHERE embedding IS NULL")
-                    remaining = int(cur.fetchone()[0])
+                    cur.execute(
+                        # content ~ '\S' means "has at least one non-whitespace
+                        # character", which matches Python's str.strip()
+                        # exactly. Postgres trim() defaults to SPACES ONLY, so a
+                        # row holding just a newline or a tab would still be
+                        # selected here, still raise EmbeddingError("empty
+                        # input"), and still fail on every run -- the very bug
+                        # this filter exists to stop.
+                        rf"SELECT count(*) FILTER (WHERE content ~ '\S'), "
+                        rf"       count(*) FILTER (WHERE content !~ '\S' OR content IS NULL) "
+                        f"FROM {t} WHERE embedding IS NULL"
+                    )
+                    row = cur.fetchone()
+                    remaining, unembeddable = int(row[0]), int(row[1])
+            if unembeddable:
+                # Surfaced, not swallowed: these are permanently un-embeddable
+                # and would otherwise be invisible now that they are skipped.
+                logger.info(
+                    "backfill %s: skipping %d row(s) with empty content "
+                    "(cannot be embedded; delete them or leave them text-only)",
+                    t, unembeddable,
+                )
             if dry_run:
-                result[t] = {"processed": 0, "succeeded": 0, "failed": 0, "remaining": remaining}
+                result[t] = {"processed": 0, "succeeded": 0, "failed": 0,
+                             "remaining": remaining, "unembeddable": unembeddable}
                 continue
 
             processed = succeeded = failed = 0
@@ -955,7 +1008,10 @@ class MemoryStore:
                 with self._get_pool().connection() as conn:
                     with conn.cursor(row_factory=dict_row) as cur:
                         cur.execute(
-                            f"SELECT id, content FROM {t} WHERE embedding IS NULL ORDER BY id LIMIT %s",
+                            f"SELECT id, content FROM {t} "
+                            f"WHERE embedding IS NULL "
+                            rf"  AND content ~ '\S' "
+                            f"ORDER BY id LIMIT %s",
                             (batch_size,),
                         )
                         rows = list(cur.fetchall())
@@ -991,10 +1047,14 @@ class MemoryStore:
 
             with self._get_pool().connection() as conn:
                 with conn.cursor() as cur:
-                    cur.execute(f"SELECT count(*) FROM {t} WHERE embedding IS NULL")
+                    cur.execute(
+                        f"SELECT count(*) FROM {t} "
+                        rf"WHERE embedding IS NULL AND content ~ '\S'"
+                    )
                     remaining = int(cur.fetchone()[0])
             result[t] = {"processed": processed, "succeeded": succeeded,
-                         "failed": failed, "remaining": remaining}
+                         "failed": failed, "remaining": remaining,
+                         "unembeddable": unembeddable}
         return result
 
     def prune_conversations(self, *, older_than_days: int, dry_run: bool = False) -> int:

{hermes_memory_pgvector-0.5.0 → hermes_memory_pgvector-0.5.2}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "hermes-memory-pgvector"
-version = "0.5.0"
+version = "0.5.2"
 description = "Postgres + pgvector memory provider plugin for hermes-agent. Multi-agent storage layer with per-minion themes, identity governance, agent attribution + delegation provenance, async writer, no LLM in the memory hot path."
 readme = "README.md"
 requires-python = ">=3.11"

hermes_memory_pgvector-0.5.2/tests/test_empty_content.py

@@ -0,0 +1,332 @@
+"""Empty-content rows must never be created, and never re-tried forever.
+
+No DB required for the write-path half; the backfill half is live-mode.
+
+IMPORTANT: point PG_TEST_DSN at a THROWAWAY database, never production. The
+backfill cases call backfill_null_embeddings(), which sweeps the WHOLE table --
+so on a shared database it would stamp every pre-existing NULL-embedding row
+with this test's constant [0.1]*768 vector. Those rows are then no longer NULL
+and a real backfill can never repair them. The `failed` assertion is table-wide
+for the same reason.
+
+Found in production. One row in `memory_entries` sat with `embedding IS NULL`
+and `length(content) = 0` -- its metadata (`old_text`, `tool_name`) showing it
+arrived through the built-in tool's `replace` path with empty new content.
+
+Two separate defects met there:
+
+  * nothing on the write path rejected empty content, so the row was created;
+  * `backfill_null_embeddings` selected `WHERE embedding IS NULL` with no
+    content filter, so every nightly sweep re-fetched it, called embed(), got
+    EmbeddingError("empty input") -- which is unconditional for empty text --
+    marked it `failed`, and moved on. Forever.
+
+The second is the damaging one. It pins `failed` above zero and makes
+`remaining == 0` unreachable, so "are there un-embedded rows?" stops being
+answerable: you cannot tell one permanently-stuck row from one new genuine
+failure. Skipping them silently would be just as bad, so they are reported
+separately as `unembeddable`.
+"""
+
+from __future__ import annotations
+
+import os
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+from hermes_pgvector import PgvectorMemoryProvider  # noqa: E402
+from hermes_pgvector.store import MemoryStore  # noqa: E402
+
+
+class _RecordingWriter:
+    def __init__(self):
+        self.items = []
+
+    def enqueue(self, **kwargs) -> bool:
+        self.items.append(kwargs)
+        return True
+
+
+def _provider():
+    p = PgvectorMemoryProvider()
+    p._healthy = True
+    p._writer = _RecordingWriter()
+    p._agent_identity = "marketing"
+    p._raw_identity = "marketing"
+    p._session_id = "sess-empty"
+    return p
+
+
+# --- write path -------------------------------------------------------------
+
+@pytest.mark.parametrize("content", ["", "   ", "\n", "\t  \n "])
+@pytest.mark.parametrize("action", ["add", "replace"])
+def test_empty_add_or_replace_is_not_mirrored(action, content):
+    p = _provider()
+    p.on_memory_write(action=action, target="memory", content=content)
+    assert p._writer.items == [], (
+        f"{action!r} with {content!r} must not be mirrored -- it creates a row "
+        "that can never be embedded and that backfill retries forever"
+    )
+
+
+def test_remove_with_empty_content_is_still_mirrored():
+    """`remove` legitimately arrives with empty content -- the removal target
+    travels in metadata as old_text, not in content -- so the empty-content
+    guard must not swallow it.
+
+    NOTE: an earlier version of this docstring said the row is targeted "via
+    old_text" and left it there, which quietly asserted a path that was in fact
+    broken downstream: _worker read item.content, not extra["old_text"], so the
+    forwarded remove deleted EVERYTHING. See the _worker tests below -- this
+    test only pins that the enqueue happens, never that it is well-formed."""
+    p = _provider()
+    p.on_memory_write(
+        action="remove", target="memory", content="",
+        metadata={"old_text": "the entry being removed"},
+    )
+    assert len(p._writer.items) == 1
+    assert p._writer.items[0]["action"] == "remove"
+
+
+def test_normal_add_is_unaffected():
+    p = _provider()
+    p.on_memory_write(action="add", target="memory", content="a real durable note")
+    assert len(p._writer.items) == 1
+    assert p._writer.items[0]["content"] == "a real durable note"
+
+
+# --- backfill ---------------------------------------------------------------
+
+@pytest.fixture
+def store():
+    dsn = os.environ.get("PG_TEST_DSN")
+    if not dsn:
+        pytest.skip("PG_TEST_DSN not set")
+    s = MemoryStore(dsn)
+    s.ensure_schema()
+    agent = "pytest-empty-" + os.urandom(4).hex()
+    yield s, agent
+    import psycopg
+    with psycopg.connect(dsn) as conn:
+        with conn.cursor() as cur:
+            cur.execute("DELETE FROM memory_entries WHERE agent_identity LIKE %s", (agent + "%",))
+            conn.commit()
+
+
+def _insert_raw(s, agent, content):
+    """Bypass add() to plant the shape production actually had: NULL embedding
+    plus empty content."""
+    with s._get_pool().connection() as conn:
+        with conn.cursor() as cur:
+            cur.execute(
+                "INSERT INTO memory_entries (agent_identity, target, content, embedding, metadata)"
+                " VALUES (%s, 'memory', %s, NULL, '{}'::jsonb) RETURNING id",
+                (agent, content),
+            )
+            rid = cur.fetchone()[0]
+            conn.commit()
+    return rid
+
+
+def test_backfill_skips_empty_content_and_reports_it(store):
+    s, agent = store
+    empty_id = _insert_raw(s, agent, "")
+    real_id = _insert_raw(s, agent, "a genuinely embeddable durable note")
+
+    seen = []
+
+    def _embed_fn(text):
+        seen.append(text)
+        return [0.1] * 768
+
+    report = s.backfill_null_embeddings(embed_fn=_embed_fn, tables=["memory_entries"])
+
+    assert "" not in seen, "the empty row must never reach embed()"
+    assert any("genuinely embeddable" in x for x in seen), "the real row must be embedded"
+    assert report["memory_entries"]["unembeddable"] >= 1, "skipped rows must be reported, not hidden"
+
+    with s._get_pool().connection() as conn:
+        with conn.cursor() as cur:
+            cur.execute("SELECT embedding IS NULL FROM memory_entries WHERE id = %s", (empty_id,))
+            assert cur.fetchone()[0] is True, "empty row is left alone, not failed"
+            cur.execute("SELECT embedding IS NULL FROM memory_entries WHERE id = %s", (real_id,))
+            assert cur.fetchone()[0] is False, "real row got embedded"
+
+
+def _scoped_backlog(s, agent):
+    """`remaining`, computed for THIS test's rows only.
+
+    backfill_null_embeddings reports table-wide numbers, but the fixture only
+    cleans its own agent_identity prefix -- so asserting on the table-wide
+    figure makes the test depend on every other row in the database and fail
+    permanently after one interrupted run. Assert the same property, scoped."""
+    with s._get_pool().connection() as conn:
+        with conn.cursor() as cur:
+            cur.execute(
+                r"SELECT count(*) FROM memory_entries "
+                r"WHERE agent_identity = %s AND embedding IS NULL AND content ~ '\S'",
+                (agent,),
+            )
+            return int(cur.fetchone()[0])
+
+
+def test_backfill_remaining_can_reach_zero_despite_an_empty_row(store):
+    """The point of the fix: an un-embeddable row must not pin the backlog
+    above zero forever, or 'is the backlog clear?' becomes unanswerable."""
+    s, agent = store
+    _insert_raw(s, agent, "")
+    _insert_raw(s, agent, "another embeddable note for the sweep")
+    assert _scoped_backlog(s, agent) == 1, "one embeddable row to start"
+
+    report = s.backfill_null_embeddings(
+        embed_fn=lambda t: [0.1] * 768, tables=["memory_entries"]
+    )
+
+    assert _scoped_backlog(s, agent) == 0, (
+        "the backlog must exclude un-embeddable rows so it can actually reach 0"
+    )
+    # Asserted directly, not as a delta against a dry-run baseline: dry_run
+    # hardcodes "failed": 0 (store.py), so such a baseline is always 0 and the
+    # comparison would be theatre. `failed` is table-wide, which is sound here
+    # only because this fixture requires a THROWAWAY database (stated in the
+    # module docstring above) -- on a shared one, a pre-existing failing row
+    # would surface here, and that is worth knowing rather than hiding.
+    assert report["memory_entries"]["failed"] == 0, (
+        "an un-embeddable row must be skipped, not counted as a failure"
+    )
+    assert report["memory_entries"]["unembeddable"] >= 1
+
+
+def test_backfill_skips_whitespace_only_content(store):
+    """Not just the empty string. Postgres trim() strips SPACES ONLY, so a row
+    holding a newline or tab used to pass the filter, reach embed(), and fail
+    forever -- the same bug for a different shape. The predicate now matches
+    Python's str.strip()."""
+    s, agent = store
+    for blank in ("", "   ", chr(10), chr(9) + chr(10) + " "):
+        _insert_raw(s, agent, blank)
+    real_id = _insert_raw(s, agent, "a real note alongside the blank ones")
+
+    seen = []
+
+    def _embed_fn(text):
+        seen.append(text)
+        return [0.1] * 768
+
+    report = s.backfill_null_embeddings(embed_fn=_embed_fn, tables=["memory_entries"])
+
+    assert all(x.strip() for x in seen), f"a blank row reached embed(): {seen!r}"
+    assert report["memory_entries"]["unembeddable"] >= 4
+    assert _scoped_backlog(s, agent) == 0
+
+    with s._get_pool().connection() as conn:
+        with conn.cursor() as cur:
+            cur.execute("SELECT embedding IS NULL FROM memory_entries WHERE id = %s", (real_id,))
+            assert cur.fetchone()[0] is False
+
+
+# ---------------------------------------------------------------------------
+# CRITICAL: the remove path must never be able to wipe a whole scope.
+#
+# store.remove() matches with `content LIKE %<old_text>%`. With an empty
+# old_text that is LIKE '%%', which matches EVERY row -- so a remove that lost
+# its target deletes the entire mirror for that (agent_identity, target)
+# instead of one entry. Verified against Postgres: DELETE ... WHERE c LIKE '%%'
+# removed all rows.
+#
+# _worker used to pass old_text=item.content. The built-in tool's remove op
+# takes old_text and leaves content empty (tools/memory_tool.py:87), and the
+# host forwards old_text through METADATA (memory_manager.notify_memory_tool_write),
+# so item.content was ALWAYS '' for a remove -- making every mirrored
+# `memory remove` a full wipe of that theme.
+# ---------------------------------------------------------------------------
+
+class _FakeStore:
+    def __init__(self):
+        self.removes = []
+
+    def remove(self, *, agent_identity, target, old_text):
+        self.removes.append(old_text)
+        return 1
+
+
+def _worker_provider():
+    p = PgvectorMemoryProvider()
+    p._healthy = True
+    p._store = _FakeStore()
+    return p
+
+
+class _Item:
+    def __init__(self, content="", extra=None):
+        self.action = "remove"
+        self.agent_identity = "marketing"
+        self.target = "memory"
+        self.content = content
+        self.extra = extra or {}
+        self.metadata = {}
+
+
+def test_worker_remove_uses_old_text_not_content():
+    """The regression: content is empty for every real remove, so reading it
+    produced LIKE '%%' and deleted the whole scope."""
+    p = _worker_provider()
+    p._worker(_Item(content="", extra={"old_text": "the entry to delete"}))
+    assert p._store.removes == ["the entry to delete"]
+
+
+def test_worker_remove_refuses_when_no_target_is_available():
+    """Belt and braces: with neither content nor old_text there is nothing to
+    match, and the store must not be called at all."""
+    for item in (_Item(content="", extra={}), _Item(content="   ", extra={"old_text": "  "})):
+        p = _worker_provider()
+        p._worker(item)
+        assert p._store.removes == [], "a target-less remove must never reach the store"
+
+
+def test_worker_remove_falls_back_to_content_when_old_text_absent():
+    """Older/other callers that put the target in content still work."""
+    p = _worker_provider()
+    p._worker(_Item(content="target in content", extra={}))
+    assert p._store.removes == ["target in content"]
+
+
+def test_store_remove_rejects_empty_old_text():
+    """Store-level guard, independent of any caller: an empty pattern is
+    LIKE '%%' and must be impossible to reach by omission."""
+    s = MemoryStore.__new__(MemoryStore)   # no connection needed; guard is first
+    for bad in ("", "   ", chr(10), chr(9) + " "):
+        with pytest.raises(ValueError, match="non-empty old_text"):
+            s.remove(agent_identity="a", target="memory", old_text=bad)
+
+
+def test_store_remove_guard_runs_before_any_db_work(store):
+    """Live: the guard must fire before touching the pool, and delete nothing."""
+    s, agent = store
+    _insert_raw(s, agent, "row one that must survive")
+    _insert_raw(s, agent, "row two that must survive")
+
+    with pytest.raises(ValueError):
+        s.remove(agent_identity=agent, target="memory", old_text="")
+
+    with s._get_pool().connection() as conn:
+        with conn.cursor() as cur:
+            cur.execute("SELECT count(*) FROM memory_entries WHERE agent_identity = %s", (agent,))
+            assert cur.fetchone()[0] == 2, "an empty remove must delete nothing"
+
+
+def test_store_remove_still_deletes_a_real_match(store):
+    s, agent = store
+    _insert_raw(s, agent, "delete me please")
+    _insert_raw(s, agent, "keep me around")
+    n = s.remove(agent_identity=agent, target="memory", old_text="delete me")
+    assert n == 1
+    with s._get_pool().connection() as conn:
+        with conn.cursor() as cur:
+            cur.execute("SELECT content FROM memory_entries WHERE agent_identity = %s", (agent,))
+            assert [r[0] for r in cur.fetchall()] == ["keep me around"]