betterdb-agent-memory 0.1.0__tar.gz

betterdb_agent_memory-0.1.0/.gitignore

@@ -0,0 +1,8 @@
+__pycache__/
+*.pyc
+*.pyo
+.venv/
+dist/
+*.egg-info/
+.ruff_cache/
+.pytest_cache/

betterdb_agent_memory-0.1.0/PKG-INFO

@@ -0,0 +1,135 @@
+Metadata-Version: 2.4
+Name: betterdb-agent-memory
+Version: 0.1.0
+Summary: Long-term memory tier for AI agents backed by Valkey Search: semantic recall with recency/importance ranking, scoped capacity eviction, and consolidation. Pairs with @betterdb/agent-cache.
+Project-URL: Repository, https://github.com/BetterDB-inc/monitor
+License: MIT
+Keywords: agent,llm,memory,opentelemetry,prometheus,redis,valkey,vector-search
+Requires-Python: >=3.11
+Requires-Dist: betterdb-agent-cache>=0.7.0
+Requires-Dist: betterdb-valkey-search-kit>=0.1.0
+Requires-Dist: opentelemetry-api>=1.20.0
+Requires-Dist: prometheus-client>=0.19.0
+Provides-Extra: dev
+Requires-Dist: opentelemetry-sdk>=1.20.0; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
+Requires-Dist: pytest>=8.0.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# @betterdb/agent-memory (Python)
+
+`betterdb-agent-memory` is the long-term memory tier for AI agents, backed by
+[Valkey Search](https://valkey.io/topics/search/). It is the Python port of
+[`@betterdb/agent-memory`](https://www.npmjs.com/package/@betterdb/agent-memory)
+and pairs with [`betterdb-agent-cache`](https://pypi.org/project/betterdb-agent-cache/)
+(the short-term llm/tool/session cache tiers).
+
+Where the cache tiers are exact-match and ephemeral, the memory tier is
+semantic and durable: it embeds content, stores it in an HNSW vector index, and
+recalls it by meaning with a composite score that blends **similarity**,
+**recency** (half-life decay), and **importance**.
+
+## Features
+
+- **Semantic recall** — KNN vector search with a tunable composite score.
+- **Scoping** — memories carry `thread_id` / `agent_id` / `namespace` / `tags`;
+  recall, forget, and consolidation all filter by scope.
+- **Reinforcement** — recalled memories bump `last_accessed_at` + `access_count`,
+  so frequently-used memories stay recallable.
+- **Capacity eviction** — `max_items_per_scope` evicts the lowest-scoring
+  memories (importance + recency) once a scope exceeds its cap.
+- **Consolidation** — fold a set of older/low-importance memories into a single
+  summary memory.
+- **Live config** — re-read `recall.threshold` / weights / `halfLifeSeconds` /
+  `maxItemsPerScope` from a Valkey hash without a restart.
+- **Observability** — OpenTelemetry spans + Prometheus metrics.
+- **Discovery** — registers a marker so BetterDB Monitor can enumerate the tier.
+
+## Installation
+
+```bash
+pip install betterdb-agent-memory
+```
+
+You also need a Valkey server with the Search module loaded (e.g.
+`valkey/valkey-bundle`) and the [`valkey`](https://pypi.org/project/valkey/)
+async client.
+
+## Quick start
+
+```python
+import valkey.asyncio as valkey
+from betterdb_agent_memory import AgentMemory, AgentMemoryOptions
+
+async def embed(text: str) -> list[float]:
+    # Replace with a real embedding model (OpenAI, sentence-transformers, ...).
+    ...
+
+async def main() -> None:
+    client = valkey.Valkey(host="localhost", port=6379)
+    agent = AgentMemory(AgentMemoryOptions(client=client, embed_fn=embed))
+    await agent.initialize()
+
+    await agent.memory.remember(
+        "User prefers dark mode and concise answers.",
+        importance=0.8,
+        tags=["preference", "ui"],
+        thread_id="t1",
+    )
+
+    hits = await agent.memory.recall("what UI settings does the user like?", thread_id="t1")
+    for hit in hits:
+        print(hit.score, hit.item.content)
+
+    # Short-term cache tiers remain available:
+    # agent.llm, agent.tool, agent.session
+
+    await agent.close()
+```
+
+## Using the memory tier standalone
+
+If you only need the memory tier, construct `MemoryStore` directly:
+
+```python
+from betterdb_agent_memory import MemoryStore
+
+store = MemoryStore(client=client, name="myapp", embed_fn=embed)
+await store.ensure_index()
+await store.remember("hello", thread_id="t1")
+hits = await store.recall("hi", thread_id="t1")
+```
+
+## API
+
+### `MemoryStore`
+
+- `await ensure_index()` — create the `{name}:mem:idx` HNSW index if absent.
+- `await remember(content, *, importance=None, tags=None, source=None, ttl=None, thread_id=None, agent_id=None, namespace=None) -> str`
+- `await recall(query, *, k=None, threshold=None, tags=None, weights=None, reinforce=None, thread_id=None, agent_id=None, namespace=None) -> list[MemoryHit]`
+- `await forget(id) -> bool`
+- `await forget_by_scope(*, thread_id=None, agent_id=None, namespace=None, tags=None) -> int`
+- `await consolidate(*, summarize, older_than_seconds=None, max_importance=None, delete_sources=None, summary_importance=None, tags=None, thread_id=None, agent_id=None, namespace=None) -> ConsolidateResult`
+- `current_config() -> MemoryConfigSnapshot`
+- `await refresh_config()`
+- `await ensure_discovery_ready()`
+- `await close()`
+
+### `AgentMemory`
+
+The batteries-included facade: an `AgentCache` (llm/tool/session) plus a
+`MemoryStore` sharing one client and name. `initialize()` creates the index and
+readies discovery for both tiers; `close()` tears both down.
+
+## Scoring
+
+`composite_score = w.similarity * similarity + w.recency * recency + w.importance * importance`
+
+where `similarity = 1 - distance / 2` (cosine distance → 0..1) and `recency`
+decays with a true half-life (`0.5` at one `half_life_seconds`). Default weights
+are `{similarity: 0.6, recency: 0.25, importance: 0.15}`, default threshold
+`0.25`, default half-life 7 days.
+
+## License
+
+MIT

betterdb_agent_memory-0.1.0/README.md

@@ -0,0 +1,117 @@
+# @betterdb/agent-memory (Python)
+
+`betterdb-agent-memory` is the long-term memory tier for AI agents, backed by
+[Valkey Search](https://valkey.io/topics/search/). It is the Python port of
+[`@betterdb/agent-memory`](https://www.npmjs.com/package/@betterdb/agent-memory)
+and pairs with [`betterdb-agent-cache`](https://pypi.org/project/betterdb-agent-cache/)
+(the short-term llm/tool/session cache tiers).
+
+Where the cache tiers are exact-match and ephemeral, the memory tier is
+semantic and durable: it embeds content, stores it in an HNSW vector index, and
+recalls it by meaning with a composite score that blends **similarity**,
+**recency** (half-life decay), and **importance**.
+
+## Features
+
+- **Semantic recall** — KNN vector search with a tunable composite score.
+- **Scoping** — memories carry `thread_id` / `agent_id` / `namespace` / `tags`;
+  recall, forget, and consolidation all filter by scope.
+- **Reinforcement** — recalled memories bump `last_accessed_at` + `access_count`,
+  so frequently-used memories stay recallable.
+- **Capacity eviction** — `max_items_per_scope` evicts the lowest-scoring
+  memories (importance + recency) once a scope exceeds its cap.
+- **Consolidation** — fold a set of older/low-importance memories into a single
+  summary memory.
+- **Live config** — re-read `recall.threshold` / weights / `halfLifeSeconds` /
+  `maxItemsPerScope` from a Valkey hash without a restart.
+- **Observability** — OpenTelemetry spans + Prometheus metrics.
+- **Discovery** — registers a marker so BetterDB Monitor can enumerate the tier.
+
+## Installation
+
+```bash
+pip install betterdb-agent-memory
+```
+
+You also need a Valkey server with the Search module loaded (e.g.
+`valkey/valkey-bundle`) and the [`valkey`](https://pypi.org/project/valkey/)
+async client.
+
+## Quick start
+
+```python
+import valkey.asyncio as valkey
+from betterdb_agent_memory import AgentMemory, AgentMemoryOptions
+
+async def embed(text: str) -> list[float]:
+    # Replace with a real embedding model (OpenAI, sentence-transformers, ...).
+    ...
+
+async def main() -> None:
+    client = valkey.Valkey(host="localhost", port=6379)
+    agent = AgentMemory(AgentMemoryOptions(client=client, embed_fn=embed))
+    await agent.initialize()
+
+    await agent.memory.remember(
+        "User prefers dark mode and concise answers.",
+        importance=0.8,
+        tags=["preference", "ui"],
+        thread_id="t1",
+    )
+
+    hits = await agent.memory.recall("what UI settings does the user like?", thread_id="t1")
+    for hit in hits:
+        print(hit.score, hit.item.content)
+
+    # Short-term cache tiers remain available:
+    # agent.llm, agent.tool, agent.session
+
+    await agent.close()
+```
+
+## Using the memory tier standalone
+
+If you only need the memory tier, construct `MemoryStore` directly:
+
+```python
+from betterdb_agent_memory import MemoryStore
+
+store = MemoryStore(client=client, name="myapp", embed_fn=embed)
+await store.ensure_index()
+await store.remember("hello", thread_id="t1")
+hits = await store.recall("hi", thread_id="t1")
+```
+
+## API
+
+### `MemoryStore`
+
+- `await ensure_index()` — create the `{name}:mem:idx` HNSW index if absent.
+- `await remember(content, *, importance=None, tags=None, source=None, ttl=None, thread_id=None, agent_id=None, namespace=None) -> str`
+- `await recall(query, *, k=None, threshold=None, tags=None, weights=None, reinforce=None, thread_id=None, agent_id=None, namespace=None) -> list[MemoryHit]`
+- `await forget(id) -> bool`
+- `await forget_by_scope(*, thread_id=None, agent_id=None, namespace=None, tags=None) -> int`
+- `await consolidate(*, summarize, older_than_seconds=None, max_importance=None, delete_sources=None, summary_importance=None, tags=None, thread_id=None, agent_id=None, namespace=None) -> ConsolidateResult`
+- `current_config() -> MemoryConfigSnapshot`
+- `await refresh_config()`
+- `await ensure_discovery_ready()`
+- `await close()`
+
+### `AgentMemory`
+
+The batteries-included facade: an `AgentCache` (llm/tool/session) plus a
+`MemoryStore` sharing one client and name. `initialize()` creates the index and
+readies discovery for both tiers; `close()` tears both down.
+
+## Scoring
+
+`composite_score = w.similarity * similarity + w.recency * recency + w.importance * importance`
+
+where `similarity = 1 - distance / 2` (cosine distance → 0..1) and `recency`
+decays with a true half-life (`0.5` at one `half_life_seconds`). Default weights
+are `{similarity: 0.6, recency: 0.25, importance: 0.15}`, default threshold
+`0.25`, default half-life 7 days.
+
+## License
+
+MIT

betterdb_agent_memory-0.1.0/RELEASE_NOTES.md

@@ -0,0 +1,55 @@
+# betterdb-agent-memory v0.1.0
+
+Python port of `@betterdb/agent-memory`. Long-term memory tier for AI agents
+backed by Valkey Search — semantic recall with recency/importance ranking,
+scoped capacity eviction, and consolidation. Pairs with `betterdb-agent-cache`.
+
+Requires Valkey 8+ with the **valkey-search** module (vector index support).
+Works with ElastiCache for Valkey, Memorystore for Valkey, and MemoryDB.
+
+Built on [`betterdb-valkey-search-kit`](https://pypi.org/project/betterdb-valkey-search-kit/)
+and [`betterdb-agent-cache`](https://pypi.org/project/betterdb-agent-cache/).
+
+---
+
+## Installation
+
+```sh
+pip install betterdb-agent-memory
+```
+
+---
+
+## What's included
+
+### MemoryStore (long-term tier)
+
+| Method | Description |
+|---|---|
+| `ensure_index()` | Create or attach to the memory vector index |
+| `remember(...)` | Persist a memory with embedding, scope, tags, and importance |
+| `recall(...)` | Semantic recall ranked by similarity, recency, and importance |
+| `recall_by_vector(...)` | KNN recall from a precomputed vector |
+| `reinforce(id)` | Bump importance / recency on an existing memory |
+| `forget(...)` | Delete memories by id or filter |
+| `consolidate(...)` | Merge and summarize related memories |
+| `get(id)` / `list(...)` | Read-only fetch and scoped, paginated listing |
+| `stats()` | Doc count, evictions, and live config |
+
+Scoped capacity eviction, live config refresh, and discovery are built in.
+
+### AgentMemory facade
+
+Convenience facade over `betterdb-agent-cache` combining the exact-match cache
+tier with the long-term memory tier.
+
+### Observability
+
+- OpenTelemetry spans on every memory operation
+- Prometheus metrics for recall latency and eviction counts
+
+---
+
+## Full changelog
+
+See the repository history for detailed changes.

betterdb_agent_memory-0.1.0/betterdb_agent_memory/__init__.py

@@ -0,0 +1,88 @@
+"""betterdb-agent-memory: long-term vector memory tier for AI agents on Valkey.
+
+Re-exports everything from ``betterdb-agent-cache`` (the short-term cache tiers)
+alongside the memory tier so the facade is a single import.
+"""
+
+from __future__ import annotations
+
+import betterdb_agent_cache as _agent_cache
+from betterdb_agent_cache import *  # noqa: F401,F403
+
+from .agent_memory import AgentMemory, AgentMemoryOptions
+from .build_recall_query import MATCH_ALL_MEMORY_QUERY
+from .composite_score import (
+    composite_score,
+    recency_decay,
+    similarity_from_distance,
+)
+from .discovery import MEMORY_CACHE_TYPE, MEMORY_CAPABILITIES, MemoryDiscovery
+from .memory_store import MemoryStore
+from .telemetry import (
+    DEFAULT_METRICS_PREFIX,
+    DEFAULT_TRACER_NAME,
+    MemoryMetrics,
+    MemoryTelemetry,
+    MemoryTelemetryOptions,
+    create_memory_telemetry,
+)
+from .types import (
+    AgentMemoryConfig,
+    AgentMemoryRecallConfig,
+    ConsolidateResult,
+    EmbedFn,
+    MemoryConfigRefreshConfig,
+    MemoryConfigSnapshot,
+    MemoryDiscoveryConfig,
+    MemoryHit,
+    MemoryItem,
+    MemoryListOptions,
+    MemoryListResult,
+    MemoryScope,
+    MemoryStats,
+    MemoryStoreClient,
+    RecallWeights,
+    SummarizeFn,
+)
+
+__all__ = [
+    # Memory tier
+    "AgentMemory",
+    "AgentMemoryOptions",
+    "AgentMemoryConfig",
+    "AgentMemoryRecallConfig",
+    "MemoryStore",
+    "MemoryDiscovery",
+    "MEMORY_CACHE_TYPE",
+    "MEMORY_CAPABILITIES",
+    # Telemetry
+    "create_memory_telemetry",
+    "DEFAULT_METRICS_PREFIX",
+    "DEFAULT_TRACER_NAME",
+    "MemoryTelemetry",
+    "MemoryTelemetryOptions",
+    "MemoryMetrics",
+    # Scoring
+    "composite_score",
+    "similarity_from_distance",
+    "recency_decay",
+    # Types
+    "EmbedFn",
+    "MemoryStoreClient",
+    "MemoryScope",
+    "MemoryItem",
+    "MemoryHit",
+    "MemoryListOptions",
+    "MemoryListResult",
+    "MemoryStats",
+    "ConsolidateResult",
+    "SummarizeFn",
+    "RecallWeights",
+    "MemoryConfigSnapshot",
+    "MemoryDiscoveryConfig",
+    "MemoryConfigRefreshConfig",
+    "MATCH_ALL_MEMORY_QUERY",
+]
+
+# Surface everything agent-cache exports so consumers need only one import.
+__all__ += list(getattr(_agent_cache, "__all__", []))

betterdb_agent_memory-0.1.0/betterdb_agent_memory/_num.py

@@ -0,0 +1,51 @@
+from __future__ import annotations
+
+import math
+import re
+
+_FLOAT_RE = re.compile(r"^[+-]?(\d+\.?\d*|\.\d+)([eE][+-]?\d+)?")
+_INT_RE = re.compile(r"^[+-]?\d+")
+
+
+def _coerce_str(value: object) -> str:
+    """Coerce a valkey reply value to text the way JS ``String()`` would.
+
+    valkey-py can hand back ``bytes`` for raw replies, so decode those instead of
+    stringifying them (``str(b'1')`` is ``"b'1'"``, which would parse as NaN).
+    """
+    if isinstance(value, bytes):
+        return value.decode()
+    return str(value)
+
+
+def js_number(value: object) -> float:
+    """Mimic JavaScript ``Number(value)`` for the string inputs we see.
+
+    Empty/whitespace-only strings become ``0`` (as in JS); unparseable strings
+    become ``NaN``. ``None`` becomes ``NaN``.
+    """
+    if value is None:
+        return math.nan
+    text = _coerce_str(value).strip()
+    if text == "":
+        return 0.0
+    try:
+        return float(text)
+    except ValueError:
+        return math.nan
+
+
+def parse_float(value: object) -> float:
+    """Mimic JavaScript ``parseFloat``: parse the leading numeric portion, else NaN."""
+    if value is None:
+        return math.nan
+    match = _FLOAT_RE.match(_coerce_str(value).strip())
+    return float(match.group()) if match else math.nan
+
+
+def parse_int(value: object) -> float:
+    """Mimic JavaScript ``parseInt(value, 10)``: parse the leading integer, else NaN."""
+    if value is None:
+        return math.nan
+    match = _INT_RE.match(_coerce_str(value).strip())
+    return int(match.group()) if match else math.nan

betterdb_agent_memory-0.1.0/betterdb_agent_memory/agent_memory.py

@@ -0,0 +1,79 @@
+from __future__ import annotations
+
+import asyncio
+from dataclasses import dataclass, field
+
+from betterdb_agent_cache import AgentCache, AgentCacheOptions
+
+from .memory_store import MemoryStore
+from .telemetry import MemoryTelemetryOptions
+from .types import AgentMemoryConfig, EmbedFn, _empty_memory_config
+
+DEFAULT_NAME = "betterdb_ac"
+
+
+@dataclass(kw_only=True)
+class AgentMemoryOptions(AgentCacheOptions):
+    """Options for the batteries-included :class:`AgentMemory` facade.
+
+    Extends :class:`AgentCacheOptions` (the three short-term cache tiers) with
+    the long-term memory tier: an ``embed_fn`` to vectorize content plus an
+    optional ``memory`` sub-config.
+    """
+
+    embed_fn: EmbedFn
+    memory: AgentMemoryConfig = field(default_factory=_empty_memory_config)
+
+
+class AgentMemory:
+    """Agent cache (llm/tool/session) plus a long-term :class:`MemoryStore` tier."""
+
+    def __init__(self, options: AgentMemoryOptions) -> None:
+        if not callable(getattr(options, "embed_fn", None)):
+            raise ValueError("AgentMemory requires an embed_fn to back the memory tier")
+
+        # The name lives on the shared options object and defaults identically in
+        # both tiers, so the cache and memory key prefixes can never drift apart.
+        name = options.name
+        self._cache = AgentCache(options)
+        self.llm = self._cache.llm
+        self.tool = self._cache.tool
+        self.session = self._cache.session
+
+        memory = options.memory
+        registry = options.telemetry.registry
+        self.memory = MemoryStore(
+            client=options.client,
+            name=name,
+            embed_fn=options.embed_fn,
+            default_threshold=memory.default_threshold,
+            weights=memory.recall.weights if memory.recall is not None else None,
+            half_life_seconds=(
+                memory.recall.half_life_seconds if memory.recall is not None else None
+            ),
+            max_items_per_scope=memory.max_items_per_scope,
+            # The facade is the batteries-included product: discover the memory
+            # tier alongside the cache tiers by default, unless explicitly disabled.
+            discovery=memory.discovery,
+            config_refresh=memory.config_refresh,
+            telemetry=MemoryTelemetryOptions(registry=registry) if registry else None,
+        )
+
+    async def initialize(self) -> None:
+        # Create the memory index before discovery so a freshly constructed facade
+        # is immediately usable for remember/recall without the caller hand-rolling
+        # the FT index. A create failure surfaces — the tier is unusable without it.
+        await self.memory.ensure_index()
+        # Surface a discovery name-collision from either tier instead of swallowing it.
+        await asyncio.gather(
+            self._cache.ensure_discovery_ready(),
+            self.memory.ensure_discovery_ready(),
+            self.memory.ensure_config_refresh_started(),
+        )
+
+    async def close(self) -> None:
+        # Tear down both tiers even if one fails, so timers and heartbeats can't leak.
+        try:
+            await self.memory.close()
+        finally:
+            await self._cache.shutdown()

betterdb_agent_memory-0.1.0/betterdb_agent_memory/build_memory_index.py

@@ -0,0 +1,60 @@
+from __future__ import annotations
+
+from .build_recall_query import VECTOR_FIELD
+
+MEMORY_INDEX_ALGORITHM = "HNSW"
+
+
+def memory_index_name(name: str) -> str:
+    return f"{name}:mem:idx"
+
+
+def memory_key_prefix(name: str) -> str:
+    return f"{name}:mem:"
+
+
+def build_memory_index_args(name: str, dims: int) -> list[str]:
+    if not isinstance(dims, int) or isinstance(dims, bool) or dims <= 0:
+        raise ValueError(f"memory index dimension must be a positive integer, got: {dims}")
+    return [
+        memory_index_name(name),
+        "ON",
+        "HASH",
+        "PREFIX",
+        "1",
+        memory_key_prefix(name),
+        "SCHEMA",
+        VECTOR_FIELD,
+        "VECTOR",
+        MEMORY_INDEX_ALGORITHM,
+        "6",
+        "TYPE",
+        "FLOAT32",
+        "DIM",
+        str(dims),
+        "DISTANCE_METRIC",
+        "COSINE",
+        "threadId",
+        "TAG",
+        "agentId",
+        "TAG",
+        "namespace",
+        "TAG",
+        "tags",
+        "TAG",
+        "SEPARATOR",
+        ",",
+        "source",
+        "TAG",
+        "importance",
+        "NUMERIC",
+        "created_at",
+        "NUMERIC",
+        "SORTABLE",
+        "last_accessed_at",
+        "NUMERIC",
+        "access_count",
+        "NUMERIC",
+        "content",
+        "TEXT",
+    ]

betterdb_agent_memory-0.1.0/betterdb_agent_memory/build_memory_record.py

@@ -0,0 +1,68 @@
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass
+
+from betterdb_valkey_search_kit import encode_float32
+
+DEFAULT_IMPORTANCE = 0.5
+
+
+@dataclass
+class MemoryWrite:
+    key: str
+    fields: list[str | bytes]
+
+
+def build_memory_record(
+    name: str,
+    id: str,
+    content: str,
+    vector: list[float],
+    *,
+    importance: float | None = None,
+    tags: list[str] | None = None,
+    source: str | None = None,
+    thread_id: str | None = None,
+    agent_id: str | None = None,
+    namespace: str | None = None,
+    now: int,
+) -> MemoryWrite:
+    imp = importance if importance is not None else DEFAULT_IMPORTANCE
+    if not isinstance(imp, (int, float)) or not math.isfinite(imp) or imp < 0 or imp > 1:
+        raise ValueError(f"importance must be a finite number in [0, 1], got: {importance}")
+
+    fields: list[str | bytes] = [
+        "content",
+        content,
+        "vector",
+        encode_float32(vector),
+        "importance",
+        str(imp),
+        "created_at",
+        str(now),
+        "last_accessed_at",
+        str(now),
+        "access_count",
+        "0",
+    ]
+
+    tag_list = tags if tags is not None else []
+    for tag in tag_list:
+        if "," in tag:
+            raise ValueError(
+                f"Tag '{tag}' must not contain a comma; tags are stored comma-separated"
+            )
+    if len(tag_list) > 0:
+        fields.extend(["tags", ",".join(tag_list)])
+
+    if thread_id is not None:
+        fields.extend(["threadId", thread_id])
+    if agent_id is not None:
+        fields.extend(["agentId", agent_id])
+    if namespace is not None:
+        fields.extend(["namespace", namespace])
+    if source is not None:
+        fields.extend(["source", source])
+
+    return MemoryWrite(key=f"{name}:mem:{id}", fields=fields)