aimemoryos 0.1.1__tar.gz

aimemoryos-0.1.1/PKG-INFO

@@ -0,0 +1,159 @@
+Metadata-Version: 2.4
+Name: aimemoryos
+Version: 0.1.1
+Summary: Add your description here
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+Requires-Dist: asyncio>=4.0.0
+Requires-Dist: duckdb>=1.5.3
+Requires-Dist: faiss-cpu>=1.14.2
+Requires-Dist: google>=3.0.0
+Requires-Dist: kuzu>=0.11.3
+Requires-Dist: loguru>=0.7.3
+Requires-Dist: numpy>=2.4.6
+Requires-Dist: openrouter>=0.9.1
+Requires-Dist: orjson>=3.11.9
+Requires-Dist: pydantic>=2.13.4
+Requires-Dist: pydantic-settings>=2.14.1
+Requires-Dist: pymupdf>=1.27.2.3
+Requires-Dist: pytest>=9.0.3
+Requires-Dist: sentence-transformers>=5.5.1
+Requires-Dist: spacy>=3.8.14
+Requires-Dist: structlog>=25.5.0
+
+# AIMemoryOS
+
+A powerful, modular, multi-store memory layer for conversational agents and LLM applications. 
+
+AIMemoryOS combines vector retrieval, graph knowledge bases, and highly-durable structured storage to give your AI agents a robust, scalable long-term memory that you can plug and play directly into your own applications.
+
+## Features
+
+- **Multi-Store Architecture**: Coordinated state across SQLite (append-only events), DuckDB (canonical relational state), FAISS (vector similarity), and KuzuDB (graph relationships).
+- **Advanced Ingestion**: End-to-end pipeline with preprocessing, PII stripping, SHA-256 deduplication, and semantic chunking.
+- **Rich Retrieval**: Context assembly from multiple sources, reranking, and trace scoring.
+- **Coordinated Pruning**: Safely `forget()` memories with guaranteed cleanup across all storage backends without violating relational integrity.
+- **Snapshotting**: Take portable, verified SQLite backups of your system's event history instantly.
+- **Hallucination Firewall**: Isolate "hypothesized" or "imagined" memories to separate tables so they don't pollute your agent's ground-truth recall.
+
+---
+
+## Installation
+
+> [!IMPORTANT]
+> **Python Version Requirement**: AIMemoryOS depends on heavy AI libraries (like SpaCy, NumPy, and PyTorch) that require pre-compiled C++ binaries. You **must** use a stable Python version (**Python 3.10, 3.11, or 3.12**) to install this package successfully. Pre-release or cutting-edge versions (like Python 3.13+) will fail to compile.
+
+AIMemoryOS relies on powerful machine learning models under the hood. Make sure you have at least 2GB of free disk space before installing, as it will download heavy dependencies like PyTorch, FAISS, and HuggingFace Transformers.
+
+### Step-by-step Installation
+
+**Step 1: Create a safe virtual environment**  
+We highly recommend using a virtual environment strictly locked to a stable Python version. We suggest using [uv](https://docs.astral.sh/uv/) for incredibly fast and foolproof environment isolation:
+```bash
+# Create a Python 3.12 virtual environment
+uv venv --python 3.12
+```
+
+**Step 2: Activate the environment**
+- **Windows (PowerShell)**: `.\.venv\Scripts\activate`
+- **Mac/Linux**: `source .venv/bin/activate`
+
+**Step 3: Install AIMemoryOS**  
+Install the SDK directly from PyPI into your isolated environment:
+```bash
+uv pip install aimemoryos
+```
+*(If you are using standard `pip`, simply run `python -m pip install aimemoryos` instead).*
+
+**Step 4: Download the SpaCy language model weights**  
+The `aimemoryos` package will automatically install the `spacy` library for you. However, you must explicitly download the actual **English NLP model weights** (which pip cannot bundle) used for entity extraction:
+
+```bash
+# If using uv:
+uv run python -m spacy download en_core_web_sm
+
+# If using standard python:
+python -m spacy download en_core_web_sm
+```
+
+---
+
+## Quickstart (Plug & Play)
+
+The easiest way to integrate AIMemoryOS into your codebase is by using the `Memory` facade. Simply import it into your application and start saving/retrieving knowledge.
+
+```python
+import asyncio
+from pathlib import Path
+from aimemoryos import Memory
+
+async def run():
+    # Initialize the high-level memory facade
+    memory = Memory()
+
+    # 1. Ingest information into the memory pipeline
+    print("Ingesting memory...")
+    await memory.save(
+        document_id="doc_001",
+        content="My favorite programming language is Python and I love pizza."
+    )
+
+    # 2. Retrieve relevant context based on semantic similarity
+    print("Retrieving context...")
+    results = await memory.retrieve(
+        query="What foods do I like?",
+        top_k=5,
+        score_threshold=0.2
+    )
+    
+    # Results include the content, metadata, and relevance score
+    for r in results:
+        print(f"[{r['score']:.2f}] {r['metadata']['content']}")
+
+    # 3. Forget a memory across all stores (DuckDB, FAISS, Graph, SQLite)
+    if results:
+        memory_id = results[0]["metadata"]["memory_id"]
+        await memory.forget(memory_id=memory_id)
+    
+    # 4. Take a verifiable snapshot of the entire event history
+    await memory.snapshot(output_path=Path("backups/memory_snapshot.sqlite"))
+
+if __name__ == "__main__":
+    asyncio.run(run())
+```
+
+---
+
+## Configuration
+
+By default, runtime databases and indexes are stored in the `.aimemoryos/` folder at the root of the installed package. You can easily override this globally if you want your data saved elsewhere:
+
+```bash
+export AIMEMORYOS_DATA_DIR="/path/to/custom/data"
+```
+
+---
+
+## Architecture / Project Layout
+
+For those interested in extending the SDK or contributing, the folder architecture is maintained as follows:
+
+- `agents/`: SDK-facing components. Contains `MemoryClient`, which is the primary unified interface to the ecosystem.
+- `main.py`: A lightweight, backwards-compatible wrapper around `MemoryClient` for immediate plug-and-play.
+- `core/runtime.py`: The composition root. Handles dependency injection, configuration, and instantiating storage/retrieval services.
+- `ingestion/`: Text preprocessing, SHA deduplication, routing, and chunking boundaries.
+- `retrieval/`: Similarity search, vector reranking, and `ContextBuilder` logic.
+- `storage/`: Highly durable persistence backends (`duckdb_store.py`, `faiss_store.py`, `sqlite_log.py`) and the `orchestrator.py`.
+- `vector/`: Embedding generation (`sentence-transformers`) and model management.
+- `graph/`: KuzuDB ontology and schema definitions.
+
+## Contributing
+
+We welcome contributions! To set up for local development:
+
+```bash
+git clone https://github.com/your-org/aimemoryos.git
+cd aimemoryos
+pip install -e .
+pytest -q
+```

aimemoryos-0.1.1/README.md

@@ -0,0 +1,136 @@
+# AIMemoryOS
+
+A powerful, modular, multi-store memory layer for conversational agents and LLM applications. 
+
+AIMemoryOS combines vector retrieval, graph knowledge bases, and highly-durable structured storage to give your AI agents a robust, scalable long-term memory that you can plug and play directly into your own applications.
+
+## Features
+
+- **Multi-Store Architecture**: Coordinated state across SQLite (append-only events), DuckDB (canonical relational state), FAISS (vector similarity), and KuzuDB (graph relationships).
+- **Advanced Ingestion**: End-to-end pipeline with preprocessing, PII stripping, SHA-256 deduplication, and semantic chunking.
+- **Rich Retrieval**: Context assembly from multiple sources, reranking, and trace scoring.
+- **Coordinated Pruning**: Safely `forget()` memories with guaranteed cleanup across all storage backends without violating relational integrity.
+- **Snapshotting**: Take portable, verified SQLite backups of your system's event history instantly.
+- **Hallucination Firewall**: Isolate "hypothesized" or "imagined" memories to separate tables so they don't pollute your agent's ground-truth recall.
+
+---
+
+## Installation
+
+> [!IMPORTANT]
+> **Python Version Requirement**: AIMemoryOS depends on heavy AI libraries (like SpaCy, NumPy, and PyTorch) that require pre-compiled C++ binaries. You **must** use a stable Python version (**Python 3.10, 3.11, or 3.12**) to install this package successfully. Pre-release or cutting-edge versions (like Python 3.13+) will fail to compile.
+
+AIMemoryOS relies on powerful machine learning models under the hood. Make sure you have at least 2GB of free disk space before installing, as it will download heavy dependencies like PyTorch, FAISS, and HuggingFace Transformers.
+
+### Step-by-step Installation
+
+**Step 1: Create a safe virtual environment**  
+We highly recommend using a virtual environment strictly locked to a stable Python version. We suggest using [uv](https://docs.astral.sh/uv/) for incredibly fast and foolproof environment isolation:
+```bash
+# Create a Python 3.12 virtual environment
+uv venv --python 3.12
+```
+
+**Step 2: Activate the environment**
+- **Windows (PowerShell)**: `.\.venv\Scripts\activate`
+- **Mac/Linux**: `source .venv/bin/activate`
+
+**Step 3: Install AIMemoryOS**  
+Install the SDK directly from PyPI into your isolated environment:
+```bash
+uv pip install aimemoryos
+```
+*(If you are using standard `pip`, simply run `python -m pip install aimemoryos` instead).*
+
+**Step 4: Download the SpaCy language model weights**  
+The `aimemoryos` package will automatically install the `spacy` library for you. However, you must explicitly download the actual **English NLP model weights** (which pip cannot bundle) used for entity extraction:
+
+```bash
+# If using uv:
+uv run python -m spacy download en_core_web_sm
+
+# If using standard python:
+python -m spacy download en_core_web_sm
+```
+
+---
+
+## Quickstart (Plug & Play)
+
+The easiest way to integrate AIMemoryOS into your codebase is by using the `Memory` facade. Simply import it into your application and start saving/retrieving knowledge.
+
+```python
+import asyncio
+from pathlib import Path
+from aimemoryos import Memory
+
+async def run():
+    # Initialize the high-level memory facade
+    memory = Memory()
+
+    # 1. Ingest information into the memory pipeline
+    print("Ingesting memory...")
+    await memory.save(
+        document_id="doc_001",
+        content="My favorite programming language is Python and I love pizza."
+    )
+
+    # 2. Retrieve relevant context based on semantic similarity
+    print("Retrieving context...")
+    results = await memory.retrieve(
+        query="What foods do I like?",
+        top_k=5,
+        score_threshold=0.2
+    )
+    
+    # Results include the content, metadata, and relevance score
+    for r in results:
+        print(f"[{r['score']:.2f}] {r['metadata']['content']}")
+
+    # 3. Forget a memory across all stores (DuckDB, FAISS, Graph, SQLite)
+    if results:
+        memory_id = results[0]["metadata"]["memory_id"]
+        await memory.forget(memory_id=memory_id)
+    
+    # 4. Take a verifiable snapshot of the entire event history
+    await memory.snapshot(output_path=Path("backups/memory_snapshot.sqlite"))
+
+if __name__ == "__main__":
+    asyncio.run(run())
+```
+
+---
+
+## Configuration
+
+By default, runtime databases and indexes are stored in the `.aimemoryos/` folder at the root of the installed package. You can easily override this globally if you want your data saved elsewhere:
+
+```bash
+export AIMEMORYOS_DATA_DIR="/path/to/custom/data"
+```
+
+---
+
+## Architecture / Project Layout
+
+For those interested in extending the SDK or contributing, the folder architecture is maintained as follows:
+
+- `agents/`: SDK-facing components. Contains `MemoryClient`, which is the primary unified interface to the ecosystem.
+- `main.py`: A lightweight, backwards-compatible wrapper around `MemoryClient` for immediate plug-and-play.
+- `core/runtime.py`: The composition root. Handles dependency injection, configuration, and instantiating storage/retrieval services.
+- `ingestion/`: Text preprocessing, SHA deduplication, routing, and chunking boundaries.
+- `retrieval/`: Similarity search, vector reranking, and `ContextBuilder` logic.
+- `storage/`: Highly durable persistence backends (`duckdb_store.py`, `faiss_store.py`, `sqlite_log.py`) and the `orchestrator.py`.
+- `vector/`: Embedding generation (`sentence-transformers`) and model management.
+- `graph/`: KuzuDB ontology and schema definitions.
+
+## Contributing
+
+We welcome contributions! To set up for local development:
+
+```bash
+git clone https://github.com/your-org/aimemoryos.git
+cd aimemoryos
+pip install -e .
+pytest -q
+```

aimemoryos-0.1.1/aimemoryos/__init__.py

@@ -0,0 +1,3 @@
+from aimemoryos.main import Memory
+
+__all__ = ['Memory']

aimemoryos-0.1.1/aimemoryos/agents/__init__.py

@@ -0,0 +1,7 @@
+"""Agents package public exports."""
+
+__all__ = [
+	"memory_client",
+	"context_builder",
+	"session_manager",
+]

aimemoryos-0.1.1/aimemoryos/agents/context_builder.py

@@ -0,0 +1,171 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Iterable
+
+from aimemoryos.retrieval.context_assembler import ContextBlock
+
+# ---------------------------------------------------------------------------
+# Defaults
+# ---------------------------------------------------------------------------
+
+DEFAULT_HEADER: str = "Relevant Memory Context"
+DEFAULT_SEPARATOR: str = "\n\n"
+DEFAULT_MAX_MEMORIES: int = 10
+
+_CHARS_PER_TOKEN: int = 4
+
+@dataclass(slots=True)
+class BuiltContext:
+    """
+    Final prompt-ready context payload returned by ContextBuilder.
+
+    Fields
+    ------
+    text             : the assembled string ready for LLM injection
+    memory_count     : number of memories included (≤ max_memories)
+    truncated        : True when source list exceeded max_memories
+    estimated_tokens : rough token count (len(text) // 4, min 1)
+    """
+
+    text: str
+    memory_count: int
+    truncated: bool
+    estimated_tokens: int
+
+
+class ContextBuilder:
+    """
+    Prompt-oriented context assembler.
+
+    Responsibilities
+    ----------------
+    - ContextBlock → prompt string formatting
+    - Memory ordering preservation (caller supplies ranked order)
+    - Optional score / metadata annotation per memory block
+    - Token budget estimation via char-count heuristic
+    - Truncation to max_memories
+
+    Does NOT:
+    - retrieve memories
+    - rerank memories
+    - access any storage backend
+    - mutate memories
+    """
+
+    def __init__(
+        self,
+        *,
+        max_memories: int = DEFAULT_MAX_MEMORIES,
+        separator: str = DEFAULT_SEPARATOR,
+        include_metadata: bool = False,
+        include_scores: bool = False,
+    ) -> None:
+        self.max_memories = max_memories
+        self.separator = separator
+        self.include_metadata = include_metadata
+        self.include_scores = include_scores
+
+    # ------------------------------------------------------------------
+    # Primary build path
+    # ------------------------------------------------------------------
+
+    def build(self, context: ContextBlock) -> BuiltContext:
+        """
+        Convert a ContextBlock into a prompt-ready BuiltContext.
+
+        Memory ordering is preserved from context.memories — the retrieval
+        engine has already applied RRF fusion and importance weighting.
+        Truncation to max_memories is applied here if the list is longer.
+
+        Each memory is rendered by _render_memory(), which appends optional
+        score and metadata lines controlled by include_scores /
+        include_metadata. The full output is prefixed with DEFAULT_HEADER.
+        """
+        selected = context.memories[: self.max_memories]
+        truncated = len(context.memories) > self.max_memories
+
+        sections: list[str] = [DEFAULT_HEADER]
+        for index, memory in enumerate(selected, start=1):
+            sections.append(self._render_memory(index=index, memory=memory))
+
+        text = self.separator.join(sections)
+        return BuiltContext(
+            text=text,
+            memory_count=len(selected),
+            truncated=truncated,
+            estimated_tokens=self._estimate_tokens(text),
+        )
+
+    # ------------------------------------------------------------------
+    # Lightweight raw-string path
+    # ------------------------------------------------------------------
+
+    def build_raw(self, memories: Iterable[str]) -> BuiltContext:
+        """
+        Assemble a BuiltContext from an iterable of pre-formatted strings.
+
+        Truncates to max_memories. No header is prepended. Useful when the
+        caller has already formatted memory strings and only needs truncation
+        and token estimation (e.g. unit tests, custom formatting pipelines).
+        """
+        all_items = list(memories)
+        truncated = len(all_items) > self.max_memories
+        selected = all_items[: self.max_memories]
+        text = self.separator.join(selected)
+        return BuiltContext(
+            text=text,
+            memory_count=len(selected),
+            truncated=truncated,
+            estimated_tokens=self._estimate_tokens(text),
+        )
+
+    # ------------------------------------------------------------------
+    # Memory rendering
+    # ------------------------------------------------------------------
+
+    def _render_memory(self, *, index: int, memory: object) -> str:
+        """
+        Format a single RetrievalCandidate into a prompt block.
+
+        Always includes:
+          [Memory N]
+          <content>
+
+        Conditionally includes (controlled by constructor flags):
+          [score=X.XXXX]          — when include_scores=True and trace present
+          [metadata={...}]        — when include_metadata=True and trace metadata present
+        """
+        lines: list[str] = []
+        lines.append(f"[Memory {index}]")
+        lines.append(getattr(memory, "content", ""))
+
+        if self.include_scores:
+            trace = getattr(memory, "trace", None)
+            if trace is not None:
+                score = getattr(trace, "final_score", None)
+                if score is not None:
+                    lines.append(f"[score={score:.4f}]")
+
+        if self.include_metadata:
+            trace = getattr(memory, "trace", None)
+            metadata = getattr(trace, "trace_metadata", None) if trace else None
+            if metadata:
+                lines.append(f"[metadata={metadata}]")
+
+        return "\n".join(lines)
+
+    # ------------------------------------------------------------------
+    # Token estimation
+    # ------------------------------------------------------------------
+
+    @staticmethod
+    def _estimate_tokens(text: str) -> int:
+        """
+        Rough token count: 1 token ≈ 4 characters (min 1).
+
+        This is a heuristic. For budget-critical prompts, run a proper
+        tokeniser (e.g. tiktoken) on BuiltContext.text before submission.
+        """
+        return max(len(text) // _CHARS_PER_TOKEN, 1)
+