alpha-avatar-plugins-memory 0.6.4__tar.gz → 0.6.5__tar.gz

alpha_avatar_plugins_memory-0.6.5/PKG-INFO

@@ -0,0 +1,155 @@
+Metadata-Version: 2.4
+Name: alpha-avatar-plugins-memory
+Version: 0.6.5
+Summary: AlphaAvatar Framework plugin for memory service
+Project-URL: Source, https://github.com/AlphaAvatar/AlphaAvatar
+License-Expression: Apache-2.0
+Keywords: AI,agents,langchain,llm,memory
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Multimedia :: Sound/Audio
+Classifier: Topic :: Multimedia :: Video
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: <3.12,>=3.11
+Requires-Dist: torchaudio>=0.10.1
+Description-Content-Type: text/markdown
+
+# 🧠 AlphaAvatar Memory Plugin
+
+The Memory Plugin provides persistent conversation, tool, and environment memory for AlphaAvatar.
+
+## 🧩 Module Overview
+
+The plugin is responsible for:
+
+* extracting durable memories from conversations and tool results;
+* building ENV memory from live perception observations;
+* attaching identity, session, evidence, and graph metadata;
+* storing memory records in local or remote vector databases;
+* retrieving relevant memories for future interactions.
+
+Memory extraction is divided into independent paths:
+
+```text
+Conversation
+    └── → Conversation Memory
+
+Tool / RAG / DeepResearch Result
+    └── → Tool Memory
+
+Visual or Audio Observation
+    └── → ENV Memory
+```
+
+Supported memory types include:
+
+| Memory Type    | Description                                                         |
+| -------------- | ------------------------------------------------------------------- |
+| `Avatar`       | Stable information about the avatar itself                          |
+| `Conversation` | User information, preferences, decisions, and interaction history   |
+| `Tools`        | Useful results produced by tools, RAG, MCP, or research             |
+| `ENV`          | Information extracted from visual or audio environment observations |
+
+## 🌍 ENV Memory
+
+ENV Memory consumes observation windows from `PerceptionRuntime`.
+
+```text
+Video / Screen / Audio Observation
+                ↓
+        PerceptionRuntime
+                ↓
+       EnvMemoryScheduler
+                ↓
+   Multimodal ENV Extraction
+                ↓
+         MemoryType.ENV
+```
+
+`EnvMemoryScheduler` manages:
+
+* periodic ENV memory updates;
+* updates triggered by user turns;
+* resetting the periodic interval after a successful triggered capture;
+* fast perception cursor commits;
+* serialized model inference;
+* pending batch merging and retry handling.
+
+ENV memory is only updated when visual or audio observations are available.
+
+User text does not independently create ENV memory. It is attached only as additional context when real environment observations are present.
+
+```text
+Visual / Audio Observation + User Text
+                    ↓
+             ENV Memory Update
+
+User Text Only
+      ↓
+No ENV Memory Update
+```
+
+Visual observations are sampled before extraction to avoid repeatedly processing similar frames. Raw audio frames are ignored; future audio support should use audio segments or semantic audio events.
+
+Heavy runtime payloads such as video frames and audio buffers are not stored directly as memory records. ENV memory stores extracted text, evidence references, and metadata.
+
+## 🕸️ Storage and Retrieval
+
+Memory records may contain:
+
+* memory text;
+* memory type;
+* object ownership;
+* session and timestamp metadata;
+* observation evidence;
+* graph nodes and links;
+* provider-specific metadata.
+
+Supported storage backends include:
+
+| Backend   | Usage                                |
+| --------- | ------------------------------------ |
+| `LanceDB` | Local vector storage and retrieval   |
+| `Qdrant`  | Remote or self-hosted vector storage |
+
+Memory retrieval supports:
+
+* semantic context search;
+* filtering by memory type, session, or object ID;
+* graph-node lookup;
+* graph-neighbor expansion;
+* identity alias resolution.
+
+Graph metadata improves retrieval across related users, speakers, faces, objects, tools, sessions, and events, while memory text remains the primary stored information.
+
+## 🔗 Module Relationships
+
+```text
+PerceptionRuntime
+    └── → ENV Memory observations
+
+Persona Plugin
+    └── → user, face, and speaker identity aliases
+
+RAG / DeepResearch / MCP
+    └── → Tool Memory
+
+Memory Plugin
+    ├── → Dynamic runtime context
+    ├── → Semantic retrieval
+    └── → Graph-aware retrieval
+```
+
+## 📦 Installation
+
+```bash
+pip install alpha-avatar-plugins-memory
+```
+
+The plugin is loaded through the AlphaAvatar configuration.

alpha_avatar_plugins_memory-0.6.5/README.md

@@ -0,0 +1,133 @@
+# 🧠 AlphaAvatar Memory Plugin
+
+The Memory Plugin provides persistent conversation, tool, and environment memory for AlphaAvatar.
+
+## 🧩 Module Overview
+
+The plugin is responsible for:
+
+* extracting durable memories from conversations and tool results;
+* building ENV memory from live perception observations;
+* attaching identity, session, evidence, and graph metadata;
+* storing memory records in local or remote vector databases;
+* retrieving relevant memories for future interactions.
+
+Memory extraction is divided into independent paths:
+
+```text
+Conversation
+    └── → Conversation Memory
+
+Tool / RAG / DeepResearch Result
+    └── → Tool Memory
+
+Visual or Audio Observation
+    └── → ENV Memory
+```
+
+Supported memory types include:
+
+| Memory Type    | Description                                                         |
+| -------------- | ------------------------------------------------------------------- |
+| `Avatar`       | Stable information about the avatar itself                          |
+| `Conversation` | User information, preferences, decisions, and interaction history   |
+| `Tools`        | Useful results produced by tools, RAG, MCP, or research             |
+| `ENV`          | Information extracted from visual or audio environment observations |
+
+## 🌍 ENV Memory
+
+ENV Memory consumes observation windows from `PerceptionRuntime`.
+
+```text
+Video / Screen / Audio Observation
+                ↓
+        PerceptionRuntime
+                ↓
+       EnvMemoryScheduler
+                ↓
+   Multimodal ENV Extraction
+                ↓
+         MemoryType.ENV
+```
+
+`EnvMemoryScheduler` manages:
+
+* periodic ENV memory updates;
+* updates triggered by user turns;
+* resetting the periodic interval after a successful triggered capture;
+* fast perception cursor commits;
+* serialized model inference;
+* pending batch merging and retry handling.
+
+ENV memory is only updated when visual or audio observations are available.
+
+User text does not independently create ENV memory. It is attached only as additional context when real environment observations are present.
+
+```text
+Visual / Audio Observation + User Text
+                    ↓
+             ENV Memory Update
+
+User Text Only
+      ↓
+No ENV Memory Update
+```
+
+Visual observations are sampled before extraction to avoid repeatedly processing similar frames. Raw audio frames are ignored; future audio support should use audio segments or semantic audio events.
+
+Heavy runtime payloads such as video frames and audio buffers are not stored directly as memory records. ENV memory stores extracted text, evidence references, and metadata.
+
+## 🕸️ Storage and Retrieval
+
+Memory records may contain:
+
+* memory text;
+* memory type;
+* object ownership;
+* session and timestamp metadata;
+* observation evidence;
+* graph nodes and links;
+* provider-specific metadata.
+
+Supported storage backends include:
+
+| Backend   | Usage                                |
+| --------- | ------------------------------------ |
+| `LanceDB` | Local vector storage and retrieval   |
+| `Qdrant`  | Remote or self-hosted vector storage |
+
+Memory retrieval supports:
+
+* semantic context search;
+* filtering by memory type, session, or object ID;
+* graph-node lookup;
+* graph-neighbor expansion;
+* identity alias resolution.
+
+Graph metadata improves retrieval across related users, speakers, faces, objects, tools, sessions, and events, while memory text remains the primary stored information.
+
+## 🔗 Module Relationships
+
+```text
+PerceptionRuntime
+    └── → ENV Memory observations
+
+Persona Plugin
+    └── → user, face, and speaker identity aliases
+
+RAG / DeepResearch / MCP
+    └── → Tool Memory
+
+Memory Plugin
+    ├── → Dynamic runtime context
+    ├── → Semantic retrieval
+    └── → Graph-aware retrieval
+```
+
+## 📦 Installation
+
+```bash
+pip install alpha-avatar-plugins-memory
+```
+
+The plugin is loaded through the AlphaAvatar configuration.

{alpha_avatar_plugins_memory-0.6.4 → alpha_avatar_plugins_memory-0.6.5}/alphaavatar/plugins/memory/__init__.py

@@ -15,6 +15,7 @@ import os
 
 from alphaavatar.agents import AvatarModule, AvatarPlugin
 from alphaavatar.agents.runtime import AvatarRuntime
+from alphaavatar.agents.runtime.inference import InferenceRunner
 
 from .log import logger
 from .memory_runtime import MemoryRuntime
@@ -55,29 +56,32 @@ class MemoryPlugin(AvatarPlugin):
             raise ImportError(f"Failed to initialize MemoryRuntime plugin: {e}") from e
 
 
-def bootstrap_inference_runners() -> None:
-    """
-    Plugin-owned runner bootstrap.
+def configure_vdb_runner(vdb_type: str | None = None) -> None:
+    vdb_type = vdb_type or os.getenv("MEMORY_VDB_TYPE")
 
-    Called by AlphaAvatar core after AvatarConfig is parsed.
-    """
-    memory_vdb_type = os.getenv("MEMORY_VDB_TYPE", "lancedb")
+    logger.info("Configuring Persona plugin with VDB type: %s", vdb_type)
 
-    if memory_vdb_type == "qdrant":
+    if vdb_type == "qdrant":
         from .runner import QdrantRunner
 
-        os.environ["MEMORY_INFERENCE_METHOD"] = QdrantRunner.INFERENCE_METHOD
-        AvatarPlugin.register_inference_runner_once(QdrantRunner)
-        return
+        method = QdrantRunner.INFERENCE_METHOD
+        InferenceRunner.register(QdrantRunner)
 
-    if memory_vdb_type == "lancedb":
+    elif vdb_type == "lancedb":
         from .runner import LanceDBRunner
 
-        os.environ["MEMORY_INFERENCE_METHOD"] = LanceDBRunner.INFERENCE_METHOD
-        AvatarPlugin.register_inference_runner_once(LanceDBRunner)
-        return
+        method = LanceDBRunner.INFERENCE_METHOD
+        InferenceRunner.register(LanceDBRunner)
 
-    logger.warning("Unsupported MEMORY_VDB_TYPE=%r", memory_vdb_type)
+    else:
+        logger.warning(
+            "Unsupported MEMORY_VDB_TYPE=%r. Expected 'qdrant' or 'lancedb'.",
+            vdb_type,
+        )
+        return None
+
+    os.environ["MEMORY_VDB_INFERENCE_METHOD"] = method
+    return None
 
 
 # Plugin register
@@ -87,8 +91,8 @@ AvatarPlugin.register_avatar_plugin(
     MemoryPlugin(),
 )
 
-# Runner bootstrap register
+# Inference Runners
 AvatarPlugin.register_inference_runner_bootstrap(
-    "alphaavatar.plugins.memory",
-    bootstrap_inference_runners,
+    "alphaavatar.plugins.memory.vdb",
+    configure_vdb_runner,
 )

alpha_avatar_plugins_memory-0.6.5/alphaavatar/plugins/memory/env_memory/__init__.py

@@ -0,0 +1,19 @@
+# Copyright 2026 AlphaAvatar project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+from .scheduler import EnvMemoryBatch, EnvMemoryScheduler
+
+__all__ = [
+    "EnvMemoryBatch",
+    "EnvMemoryScheduler",
+]