alpha-avatar-plugins-memory 0.6.2__tar.gz → 0.6.4__tar.gz

{alpha_avatar_plugins_memory-0.6.2 → alpha_avatar_plugins_memory-0.6.4}/.gitignore

@@ -130,7 +130,6 @@ celerybeat.pid
 .env.personal
 .env.prod
 .venv
-env/
 venv/
 ENV/
 env.bak/

alpha_avatar_plugins_memory-0.6.4/PKG-INFO

@@ -0,0 +1,242 @@
+Metadata-Version: 2.4
+Name: alpha-avatar-plugins-memory
+Version: 0.6.4
+Summary: AlphaAvatar Framework plugin for memory service
+Project-URL: Source, https://github.com/AlphaAvatar/AlphaAvatar
+License-Expression: Apache-2.0
+Keywords: AI,agents,langchain,llm,memory
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Multimedia :: Sound/Audio
+Classifier: Topic :: Multimedia :: Video
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: <3.12,>=3.11
+Requires-Dist: torchaudio>=0.10.1
+Description-Content-Type: text/markdown
+
+# 🧠 Memory Plugin for AlphaAvatar
+
+> Give AlphaAvatar the ability to **remember you, its tools, and its environment** — across conversations, sessions, and time.
+
+---
+
+## 🤔 What is the Memory Plugin?
+
+Imagine talking to an assistant that forgets everything the moment you close the app. Frustrating, right?
+
+The **Memory Plugin** gives AlphaAvatar a persistent memory layer. It can:
+
+- Remember your **preferences, goals, and important facts**
+- Recall **past conversations** and previous decisions
+- Learn from **tool usage**, research, and retrieved documents
+- Build **environment memories** from live visual observations
+- Retrieve related memories through **semantic and graph-aware search**
+
+Think of it as AlphaAvatar's long-term notebook — important information is extracted, organized, stored, and brought back when it becomes useful.
+
+---
+
+## 💡 How Does it Work? (Simple Flow)
+
+```text
+Conversation / Tool Result / Visual Observation
+                    ↓
+AlphaAvatar extracts useful memory updates
+                    ↓
+Runtime adds identity, session, graph, and evidence metadata
+                    ↓
+Memories are stored in local or remote vector storage
+                    ↓
+Relevant memories are retrieved in future turns
+                    ↓
+AlphaAvatar responds with better continuity and grounding
+```
+
+Conversation and tool memory work automatically in the background.
+Online ENV memory is enabled when a multimodal `env_delta_task` is configured.
+
+---
+
+## ✨ Features
+
+### 🌍 Persistent Memory
+
+AlphaAvatar maintains memories beyond a single chat session.
+
+This includes:
+
+- Things **you** told it
+- Important **conversation decisions**
+- Results from **MCP, RAG, and DeepResearch**
+- Useful observations from the **camera, screen, or surrounding environment**
+- Links between people, objects, sessions, tools, and events
+
+### ⚡ Runtime Memory Updates
+
+Memory can update while a session is active instead of waiting until the conversation ends.
+
+Conversation, tool, and ENV memory use separate extraction paths, while the runtime controls timestamps, ownership, evidence, graph structure, and session metadata.
+
+### 👁️ Online ENV Memory
+
+The Memory plugin can consume ordered observation windows from `PerceptionRuntime`.
+
+```text
+Video / Screen Observations
+            ↓
+PerceptionRuntime Window
+            ↓
+Multimodal ENV Extraction Task
+            ↓
+EnvMemoryDelta
+            ↓
+MemoryType.ENV
+```
+
+ENV memory prefers annotated visual evidence when available and falls back to raw frames. Heavy runtime payloads such as video frames are never written directly into memory storage.
+
+### 🕸️ Graph-aware Retrieval
+
+Memories can include graph nodes and links for entities such as:
+
+- users
+- faces and speakers
+- objects
+- sessions
+- tools
+- locations
+- events
+
+Graph nodes act as retrieval anchors, while memory text remains the source of truth.
+
+---
+
+## 📦 What Kind of Things Does AlphaAvatar Remember?
+
+| Type | Example |
+|------|---------|
+| 👤 Personal facts | “My name is Alex and I live in New York.” |
+| ❤️ Preferences | “I prefer short, direct answers.” |
+| 🗓️ Conversation memory | A decision or topic from an earlier session |
+| 🔍 Research results | Findings from a DeepResearch task |
+| 📄 Document knowledge | Useful information retrieved through RAG |
+| 🛠️ Tool interactions | Results from MCP tools such as Gmail or Notion |
+| 👁️ ENV memory | “A person placed a red cup on the desk.” |
+| 🕸️ Graph associations | A user, object, tool, and event linked across memories |
+
+---
+
+## 🔧 Installation
+
+```bash
+pip install alpha-avatar-plugins-memory
+```
+
+The plugin is then loaded through the AlphaAvatar configuration.
+
+To enable online ENV memory, configure a multimodal provider task such as:
+
+```yaml
+env_delta_task: memory.env_delta
+```
+
+If no ENV extraction task is configured, the rest of the Memory plugin continues to work normally.
+
+---
+
+## 🗄️ How Memories are Stored
+
+AlphaAvatar stores memories as structured records with searchable embeddings and runtime-owned metadata.
+
+Memory records can include:
+
+- memory text
+- memory type
+- object ownership
+- timestamps and session IDs
+- evidence references
+- graph nodes and links
+- additional runtime metadata
+
+### Supported Backends
+
+| Backend | What it Does |
+|---------|--------------|
+| **LanceDB** | Local vector storage and graph-aware retrieval (default) |
+| **Qdrant** | Remote or self-hosted vector storage |
+| **Provider Gateway** | Routes extraction tasks to configured LLM or VLM providers |
+
+By default, AlphaAvatar can run with local LanceDB storage. Raw camera frames, audio buffers, and other heavy runtime payloads are not persisted as memory items.
+
+---
+
+## 🔗 How Memory Connects to Other Modules
+
+```text
+PerceptionRuntime
+      └── → ENV Memory         (visual and future audio observations)
+
+Persona Plugin
+      └── → Identity Aliases   (face / speaker / user relationships)
+
+RAG / DeepResearch / MCP
+      └── → Tool Memory        (documents, research, and actions)
+
+Memory Plugin
+      ├── → Avatar Runtime     (dynamic per-turn context)
+      ├── → Graph Retrieval    (entity and relationship lookup)
+      ├── → Reflection*        (behavior improvement)
+      └── → Planning*          (goals and reminders)
+
+* Planned
+```
+
+---
+
+## 🙋 Common Questions
+
+**Q: Does memory work across sessions?**
+Yes. Stored memories can be retrieved in later sessions.
+
+**Q: Does ENV memory save my raw camera stream?**
+No. Raw runtime frames are not stored as memory records. The configured multimodal model extracts useful environment memories from selected observation windows.
+
+**Q: Is ENV memory always enabled?**
+No. It is optional and only runs when `env_delta_task` is configured.
+
+**Q: Can memory distinguish different users?**
+Memory supports multi-object ownership and graph aliases. Persona can resolve temporary face or speaker identities to stable users.
+
+**Q: Can I see or delete individual memories?**
+Fine-grained inspection, correction, export, and deletion controls are still being developed.
+
+**Q: Is my data private?**
+Local LanceDB storage keeps memory data on your own machine unless you configure a remote backend or external model provider.
+
+---
+
+## 🚀 Coming Soon
+
+| Feature | Description |
+|---------|-------------|
+| 🎤 Audio ENV Memory | Extract useful events and context from continuous audio streams |
+| 🔍 Rich Visual-history Search | Search by objects, events, identities, and time ranges |
+| 🧩 Multi-annotation Fusion | Combine face, speaker, object, action, and scene annotations |
+| 🔒 Memory Privacy Controls | Inspect, edit, export, and delete memories |
+| 👥 Stronger Multi-user Memory | Better separation and merging across shared sessions |
+| 🧠 Reflection & Planning | Use memory for longer-term behavior improvement and goals |
+
+---
+
+## 📚 Related Links
+
+- [AlphaAvatar ROADMAP — Memory Section](https://github.com/AlphaAvatar/AlphaAvatar/blob/main/ROADMAP.md#-memory)
+- [Persona Plugin](https://github.com/AlphaAvatar/AlphaAvatar/blob/main/avatar-plugins/avatar-plugins-persona/README.md)
+- [LanceDB Documentation](https://lancedb.com/docs/)
+- [Qdrant Documentation](https://qdrant.tech/documentation/)

alpha_avatar_plugins_memory-0.6.4/README.md

@@ -0,0 +1,220 @@
+# 🧠 Memory Plugin for AlphaAvatar
+
+> Give AlphaAvatar the ability to **remember you, its tools, and its environment** — across conversations, sessions, and time.
+
+---
+
+## 🤔 What is the Memory Plugin?
+
+Imagine talking to an assistant that forgets everything the moment you close the app. Frustrating, right?
+
+The **Memory Plugin** gives AlphaAvatar a persistent memory layer. It can:
+
+- Remember your **preferences, goals, and important facts**
+- Recall **past conversations** and previous decisions
+- Learn from **tool usage**, research, and retrieved documents
+- Build **environment memories** from live visual observations
+- Retrieve related memories through **semantic and graph-aware search**
+
+Think of it as AlphaAvatar's long-term notebook — important information is extracted, organized, stored, and brought back when it becomes useful.
+
+---
+
+## 💡 How Does it Work? (Simple Flow)
+
+```text
+Conversation / Tool Result / Visual Observation
+                    ↓
+AlphaAvatar extracts useful memory updates
+                    ↓
+Runtime adds identity, session, graph, and evidence metadata
+                    ↓
+Memories are stored in local or remote vector storage
+                    ↓
+Relevant memories are retrieved in future turns
+                    ↓
+AlphaAvatar responds with better continuity and grounding
+```
+
+Conversation and tool memory work automatically in the background.
+Online ENV memory is enabled when a multimodal `env_delta_task` is configured.
+
+---
+
+## ✨ Features
+
+### 🌍 Persistent Memory
+
+AlphaAvatar maintains memories beyond a single chat session.
+
+This includes:
+
+- Things **you** told it
+- Important **conversation decisions**
+- Results from **MCP, RAG, and DeepResearch**
+- Useful observations from the **camera, screen, or surrounding environment**
+- Links between people, objects, sessions, tools, and events
+
+### ⚡ Runtime Memory Updates
+
+Memory can update while a session is active instead of waiting until the conversation ends.
+
+Conversation, tool, and ENV memory use separate extraction paths, while the runtime controls timestamps, ownership, evidence, graph structure, and session metadata.
+
+### 👁️ Online ENV Memory
+
+The Memory plugin can consume ordered observation windows from `PerceptionRuntime`.
+
+```text
+Video / Screen Observations
+            ↓
+PerceptionRuntime Window
+            ↓
+Multimodal ENV Extraction Task
+            ↓
+EnvMemoryDelta
+            ↓
+MemoryType.ENV
+```
+
+ENV memory prefers annotated visual evidence when available and falls back to raw frames. Heavy runtime payloads such as video frames are never written directly into memory storage.
+
+### 🕸️ Graph-aware Retrieval
+
+Memories can include graph nodes and links for entities such as:
+
+- users
+- faces and speakers
+- objects
+- sessions
+- tools
+- locations
+- events
+
+Graph nodes act as retrieval anchors, while memory text remains the source of truth.
+
+---
+
+## 📦 What Kind of Things Does AlphaAvatar Remember?
+
+| Type | Example |
+|------|---------|
+| 👤 Personal facts | “My name is Alex and I live in New York.” |
+| ❤️ Preferences | “I prefer short, direct answers.” |
+| 🗓️ Conversation memory | A decision or topic from an earlier session |
+| 🔍 Research results | Findings from a DeepResearch task |
+| 📄 Document knowledge | Useful information retrieved through RAG |
+| 🛠️ Tool interactions | Results from MCP tools such as Gmail or Notion |
+| 👁️ ENV memory | “A person placed a red cup on the desk.” |
+| 🕸️ Graph associations | A user, object, tool, and event linked across memories |
+
+---
+
+## 🔧 Installation
+
+```bash
+pip install alpha-avatar-plugins-memory
+```
+
+The plugin is then loaded through the AlphaAvatar configuration.
+
+To enable online ENV memory, configure a multimodal provider task such as:
+
+```yaml
+env_delta_task: memory.env_delta
+```
+
+If no ENV extraction task is configured, the rest of the Memory plugin continues to work normally.
+
+---
+
+## 🗄️ How Memories are Stored
+
+AlphaAvatar stores memories as structured records with searchable embeddings and runtime-owned metadata.
+
+Memory records can include:
+
+- memory text
+- memory type
+- object ownership
+- timestamps and session IDs
+- evidence references
+- graph nodes and links
+- additional runtime metadata
+
+### Supported Backends
+
+| Backend | What it Does |
+|---------|--------------|
+| **LanceDB** | Local vector storage and graph-aware retrieval (default) |
+| **Qdrant** | Remote or self-hosted vector storage |
+| **Provider Gateway** | Routes extraction tasks to configured LLM or VLM providers |
+
+By default, AlphaAvatar can run with local LanceDB storage. Raw camera frames, audio buffers, and other heavy runtime payloads are not persisted as memory items.
+
+---
+
+## 🔗 How Memory Connects to Other Modules
+
+```text
+PerceptionRuntime
+      └── → ENV Memory         (visual and future audio observations)
+
+Persona Plugin
+      └── → Identity Aliases   (face / speaker / user relationships)
+
+RAG / DeepResearch / MCP
+      └── → Tool Memory        (documents, research, and actions)
+
+Memory Plugin
+      ├── → Avatar Runtime     (dynamic per-turn context)
+      ├── → Graph Retrieval    (entity and relationship lookup)
+      ├── → Reflection*        (behavior improvement)
+      └── → Planning*          (goals and reminders)
+
+* Planned
+```
+
+---
+
+## 🙋 Common Questions
+
+**Q: Does memory work across sessions?**
+Yes. Stored memories can be retrieved in later sessions.
+
+**Q: Does ENV memory save my raw camera stream?**
+No. Raw runtime frames are not stored as memory records. The configured multimodal model extracts useful environment memories from selected observation windows.
+
+**Q: Is ENV memory always enabled?**
+No. It is optional and only runs when `env_delta_task` is configured.
+
+**Q: Can memory distinguish different users?**
+Memory supports multi-object ownership and graph aliases. Persona can resolve temporary face or speaker identities to stable users.
+
+**Q: Can I see or delete individual memories?**
+Fine-grained inspection, correction, export, and deletion controls are still being developed.
+
+**Q: Is my data private?**
+Local LanceDB storage keeps memory data on your own machine unless you configure a remote backend or external model provider.
+
+---
+
+## 🚀 Coming Soon
+
+| Feature | Description |
+|---------|-------------|
+| 🎤 Audio ENV Memory | Extract useful events and context from continuous audio streams |
+| 🔍 Rich Visual-history Search | Search by objects, events, identities, and time ranges |
+| 🧩 Multi-annotation Fusion | Combine face, speaker, object, action, and scene annotations |
+| 🔒 Memory Privacy Controls | Inspect, edit, export, and delete memories |
+| 👥 Stronger Multi-user Memory | Better separation and merging across shared sessions |
+| 🧠 Reflection & Planning | Use memory for longer-term behavior improvement and goals |
+
+---
+
+## 📚 Related Links
+
+- [AlphaAvatar ROADMAP — Memory Section](https://github.com/AlphaAvatar/AlphaAvatar/blob/main/ROADMAP.md#-memory)
+- [Persona Plugin](https://github.com/AlphaAvatar/AlphaAvatar/blob/main/avatar-plugins/avatar-plugins-persona/README.md)
+- [LanceDB Documentation](https://lancedb.com/docs/)
+- [Qdrant Documentation](https://qdrant.tech/documentation/)

{alpha_avatar_plugins_memory-0.6.2 → alpha_avatar_plugins_memory-0.6.4}/alphaavatar/plugins/memory/__init__.py

@@ -14,7 +14,7 @@
 import os
 
 from alphaavatar.agents import AvatarModule, AvatarPlugin
-from alphaavatar.agents.runtime import SessionRuntime
+from alphaavatar.agents.runtime import AvatarRuntime
 
 from .log import logger
 from .memory_runtime import MemoryRuntime
@@ -33,7 +33,8 @@ class MemoryPlugin(AvatarPlugin):
 
     def get_plugin(
         self,
-        session_runtime: SessionRuntime,
+        runtime: AvatarRuntime,
+        avatar_id: str,
         memory_search_context: int,
         memory_recall_num: int,
         maximum_memory_num: int,
@@ -43,7 +44,8 @@ class MemoryPlugin(AvatarPlugin):
     ) -> MemoryRuntime:
         try:
             return MemoryRuntime(
-                session_runtime=session_runtime,
+                runtime=runtime,
+                avatar_id=avatar_id,
                 memory_search_context=memory_search_context,
                 memory_recall_num=memory_recall_num,
                 maximum_memory_num=maximum_memory_num,

alpha_avatar_plugins_memory-0.6.4/alphaavatar/plugins/memory/graph/__init__.py

@@ -0,0 +1,24 @@
+# Copyright 2026 AlphaAvatar project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from .graph_alias import save_graph_aliases
+from .graph_builder import build_graph_from_mentions
+from .graph_lookup import GraphLookup
+from .graph_store import save_memory_graph_stubs
+
+__all__ = [
+    "save_graph_aliases",
+    "build_graph_from_mentions",
+    "GraphLookup",
+    "save_memory_graph_stubs",
+]

alpha_avatar_plugins_memory-0.6.4/alphaavatar/plugins/memory/graph/graph_alias.py

@@ -0,0 +1,144 @@
+# Copyright 2026 AlphaAvatar project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+import json
+import pathlib
+from typing import Any
+
+
+def _json_dumps(value: Any) -> str:
+    return json.dumps(value, ensure_ascii=False, default=str)
+
+
+def _json_loads(line: str) -> dict[str, Any] | None:
+    try:
+        data = json.loads(line)
+        return data if isinstance(data, dict) else None
+    except Exception:
+        return None
+
+
+def _read_alias_map(path: pathlib.Path) -> dict[str, dict[str, Any]]:
+    if not path.exists():
+        return {}
+
+    out: dict[str, dict[str, Any]] = {}
+
+    for line in path.read_text(encoding="utf-8").splitlines():
+        line = line.strip()
+        if not line:
+            continue
+
+        row = _json_loads(line)
+        if not row:
+            continue
+
+        alias_key = str(row.get("alias_key", "")).strip()
+        canonical_key = str(row.get("canonical_key", "")).strip()
+
+        if not alias_key or not canonical_key:
+            continue
+
+        out[alias_key] = row
+
+    return out
+
+
+def _write_aliases(path: pathlib.Path, aliases: dict[str, dict[str, Any]]) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+
+    rows = sorted(
+        aliases.values(),
+        key=lambda x: (
+            str(x.get("canonical_key", "")),
+            str(x.get("alias_key", "")),
+        ),
+    )
+
+    text = "\n".join(_json_dumps(row) for row in rows)
+    path.write_text(text + ("\n" if text else ""), encoding="utf-8")
+
+
+def save_graph_aliases(
+    *,
+    graph_path: str | pathlib.Path,
+    aliases: list[dict[str, Any]],
+) -> dict[str, Any]:
+    graph_path = pathlib.Path(graph_path)
+    aliases_path = graph_path / "aliases.jsonl"
+
+    alias_map = _read_alias_map(aliases_path)
+
+    updated = 0
+    skipped = 0
+    conflicts: list[dict[str, Any]] = []
+
+    for alias in aliases:
+        alias_key = str(alias.get("alias_key", "")).strip()
+        canonical_key = str(alias.get("canonical_key", "")).strip()
+
+        if not alias_key or not canonical_key or alias_key == canonical_key:
+            skipped += 1
+            continue
+
+        # local detector keys must already be scoped before aliasing
+        if alias_key.startswith(("face:", "voice:", "object:")) and ":local:" not in alias_key:
+            skipped += 1
+            conflicts.append(
+                {
+                    "alias_key": alias_key,
+                    "canonical_key": canonical_key,
+                    "reason": "unscoped_local_key",
+                }
+            )
+            continue
+
+        old = alias_map.get(alias_key)
+
+        if old:
+            old_canonical = str(old.get("canonical_key", "")).strip()
+            old_weight = float(old.get("weight", 0.0))
+            new_weight = float(alias.get("weight", 0.0))
+
+            # If conflict, only replace when new confidence is higher.
+            if old_canonical and old_canonical != canonical_key and new_weight < old_weight:
+                conflicts.append(
+                    {
+                        "alias_key": alias_key,
+                        "old_canonical_key": old_canonical,
+                        "new_canonical_key": canonical_key,
+                        "old_weight": old_weight,
+                        "new_weight": new_weight,
+                        "reason": "lower_confidence_conflict",
+                    }
+                )
+                skipped += 1
+                continue
+
+            merged = dict(old)
+            merged.update(alias)
+            alias_map[alias_key] = merged
+        else:
+            alias_map[alias_key] = alias
+
+        updated += 1
+
+    _write_aliases(aliases_path, alias_map)
+
+    return {
+        "aliases_file": str(aliases_path),
+        "aliases": len(alias_map),
+        "updated": updated,
+        "skipped": skipped,
+        "conflicts": conflicts,
+    }