fragmented-memory 1.0.0__tar.gz

fragmented_memory-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 j-zly
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

fragmented_memory-1.0.0/PKG-INFO

@@ -0,0 +1,247 @@
+Metadata-Version: 2.4
+Name: fragmented-memory
+Version: 1.0.0
+Summary: Fragmented memory system for Hermes Agent — automatically retrieve relevant memory fragments and inject them into conversation context
+Author: j-zly
+License: MIT
+Keywords: hermes-agent,memory,rag,redisearch,llm
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: redis>=5.0
+Requires-Dist: jieba>=0.42
+Dynamic: license-file
+
+# Fragmented Memory Plugin for Hermes Agent
+
+The Fragmented Memory system automatically retrieves relevant memory fragments and injects them into the conversation context for each dialogue.
+
+```text
+User: "How did we set up that React project structure last time?"
+                      ↓
+         Fragmented Memory System     ← Redis + RediSearch
+                      ↓
+    ┌─────────────────────────────────────┐
+    │  [1] User prefers TypeScript + Vite    │
+    │  [2] Previous projects used pinia state management │
+    │  [3] Backend suggested using .NET 10 implementation       │
+    └─────────────────────────────────────┘
+                      ↓
+         Model directly uses fragments to answer
+```
+
+## Features
+
+- **Semantic Splitting** — auto-split conversations into standalone fragments
+- **BM25 Full-Text Search** — works out of the box with no external API
+- **Optional Vector Search** — KNN via RediSearch (OpenAI / DashScope embedder)
+- **Time Decay** — newer fragments rank higher (60-day half-life configurable)
+- **Sentiment Weighting** — emotional fragments get priority
+- **User Feedback** — mark fragments useful/useless to improve ranking
+- **Hot Topic Boost** — frequently discussed topics rank higher
+- **Auto Sync** — every turn is archived automatically, memory tool writes are synced
+- **Auto Maintenance** — consolidation + selective forgetting to keep storage tidy
+
+## Requirements
+
+- **Python 3.10+**
+- **Hermes Agent 0.12+** — provides `MemoryProvider` interface
+- **Redis 7+** — with RediSearch module (v2.6+)
+- **jieba** — Chinese tokenization (auto-installed)
+- **Embedding API** (optional) — OpenAI / DashScope / any compatible `/v1/embeddings` service
+
+## Installation
+
+```bash
+pip install fragmented-memory
+```
+
+Or install directly from GitHub:
+
+```bash
+pip install git+https://github.com/j-zly/fragmented-memory.git
+```
+
+## Configuration
+
+Configuration precedence (high to low): **Environment variables > JSON config file > config.yaml inline > defaults**
+
+### 1. Create Redis Index (first-time usage)
+
+The code will auto-create (`ensure_index()`), or execute manually:
+
+```bash
+redis-cli FT.CREATE idx:memories ON HASH PREFIX 1 "memory:frag:" SCHEMA \
+    content TEXT WEIGHT 1 \
+    tags TAG SEPARATOR "," \
+    category TAG SEPARATOR "," \
+    source TEXT WEIGHT 1 \
+    created TEXT WEIGHT 0 \
+    fragment_type TAG SEPARATOR "," \
+    embed_bin VECTOR FLAT 6 TYPE FLOAT32 DIM 1536 DISTANCE_METRIC COSINE
+```
+
+> Dimension (DIM) is dynamically adjusted based on the embedding model used, default 1536.
+> For Docker: `docker run -d --name redis-stack -p 6379:6379 redis/redis-stack:latest`
+
+### 2. Hermes Configuration
+
+Enable in `~/.hermes/config.yaml`:
+
+```yaml
+memory:
+  provider: fragmented
+```
+
+Detailed configuration is recommended in a JSON config file (not embedded in config.yaml):
+
+`~/.config/fragmented-memory/config.json`:
+
+```json
+{
+  "redis_host": "127.0.0.1",
+  "redis_port": 6379,
+  "top_k": 5,
+  "candidate_k": 10,
+  "embedder": {
+    "provider": "dashscope",
+    "api_key": "sk-xxx",
+    "model": "text-embedding-v2"
+  }
+}
+```
+
+If `embedder` is not configured, only BM25 full-text search mode will be used.
+
+Also supports environment variable configuration (highest precedence):
+
+```bash
+export FRAGMENTED_REDIS_HOST=127.0.0.1
+export FRAGMENTED_REDIS_PORT=6379
+export FRAGMENTED_TOP_K=5
+export FRAGMENTED_EMBEDDER=dashscope
+export FRAGMENTED_EMBEDDER_MODEL=text-embedding-v2
+export OPENAI_API_KEY=sk-xxx        # embedder API key
+```
+
+### 3. Restart Gateway
+
+```bash
+# For CLI mode, restart session is sufficient
+# For Gateway mode, restart the process
+```
+
+## Configuration Reference
+
+| Config Item | Environment Variable | Default Value | Description |
+|-------------|---------------------|---------------|-------------|
+| `redis_host` | `FRAGMENTED_REDIS_HOST` | `127.0.0.1` | Redis address |
+| `redis_port` | `FRAGMENTED_REDIS_PORT` | `6379` | Redis port |
+| `top_k` | `FRAGMENTED_TOP_K` | `5` | Number of final fragments returned |
+| `candidate_k` | `FRAGMENTED_CANDIDATE_K` | `10` | Candidate fragments count (for KNN) |
+| `tag_filter` | `FRAGMENTED_TAG_FILTER` | `""` | Tag filtering (comma-separated) |
+| `bm25_limit` | `FRAGMENTED_BM25_LIMIT` | `10` | BM25 search candidate count |
+| `decay_half_days` | `FRAGMENTED_DECAY_HALF_DAYS` | `60` | Time decay half-life (days) |
+| `embed_cache_ttl` | `FRAGMENTED_EMBED_CACHE_TTL` | `3600` | Embedding cache TTL (seconds) |
+| `sentiment_boost_positive` | — | `1.5` | Positive fragment weight multiplier |
+| `sentiment_boost_negative` | — | `1.3` | Negative fragment weight multiplier |
+| `feedback_positive_boost` | — | `1.3` | Positive feedback bonus weight |
+| `feedback_negative_penalty` | — | `0.5` | Negative feedback penalty coefficient |
+| `hot_topic_boost` | — | `1.2` | Hot topic weighting multiplier |
+| `embedder.provider` | `FRAGMENTED_EMBEDDER` | `openai` | `openai` / `dashscope` |
+| `embedder.api_key` | `OPENAI_API_KEY` | — | Embedding API key |
+| `embedder.base_url` | `FRAGMENTED_EMBEDDER_URL` | `https://api.openai.com/v1` | API endpoint |
+| `embedder.model` | `FRAGMENTED_EMBEDDER_MODEL` | `text-embedding-3-small` | Embedding model name |
+| `consolidate_min_group` | — | `2` | Minimum fragments to trigger consolidation |
+| `consolidate_max_age_hours` | — | `72` | Minimum age (hours) before fragments can be consolidated |
+| `forget_max_age_days` | — | `30` | Number of days before fragments might be forgotten |
+| `forget_dry_run` | — | `true` | Safe mode for forgetting: only count, don't delete |
+| `hot_topic_decay_half_days` | — | `30` | Hot topic time decay half-life (days) |
+| `emotion_intensity_factor` | — | `0.4` | Emotion intensity → weight coefficient (0=disabled, 1=max) |
+| `attention_boost_max` | — | `1.5` | Maximum attention weighting value |
+| `attention_base_increment` | — | `2.0` | Base attention increment per mention |
+| `attention_emotion_factor` | — | `1.5` | Emotion intensity amplification factor for attention |
+
+> `sentiment_*`, `feedback_*`, `hot_topic_*` and other ranking weight parameters currently only support configuration through JSON config file, not environment variables. Set to `1.0` to disable the effect of that dimension.
+
+### Embedding Models and Dimensions
+
+| Model | Dimensions |
+|-------|------------|
+| OpenAI text-embedding-3-small | 1536 |
+| OpenAI text-embedding-3-large | 3072 |
+| OpenAI text-embedding-ada-002 | 1536 |
+| DashScope text-embedding-v2 | 1536 |
+| DashScope text-embedding-v3 | 1024 |
+
+Dimensions are automatically detected, switching models doesn't require reconfiguration.
+
+### Synonym Table
+
+Stored in Redis Hash `fragmented:synonyms`, expanded at search time to improve recall:
+
+```bash
+redis-cli HSET fragmented:synonyms setup '["install","configure","deploy","setup"]'
+redis-cli HSET fragmented:synonyms fix '["fix","modify","correct","repair","solve"]'
+```
+
+## Verification
+
+Check logs after startup:
+
+```
+Memory provider 'fragmented' registered (0 tools)
+fragmented: connected (session=xxx, top_k=5, tag_filter=(none))
+fragmented: BM25-only mode (no embedder configured)
+```
+
+## Architecture
+
+```
+┌────────────────────────────────────────────────────────┐
+│                    User sends message                  │
+└──────────────────┬─────────────────────────────────────┘
+                   │
+         ┌─────────▼─────────┐
+         │   prefetch()       │  ← Automatically triggered
+         │   ↓                │
+         │  BM25 Full-Text Search │  ← Default, zero cost
+         │  (KNN Vector search) │  ← Optional (needs embedder)
+         │   ↓                │
+         │  Six-dimensional Re-ranking │  ← Similarity × Time decay
+         │                    │    × Emotion × Feedback × Hot Topic × Attention
+         │   ↓                │
+         │  Top N Injected into Context   │
+         └─────────┬─────────┘
+                   │
+         ┌─────────▼─────────┐
+         │   Model Response   │  ← Fragments can be used as reference
+         └───────────────────┘
+                   │
+         ┌─────────▼─────────┐
+         │   sync_turn()      │  ← Automatically archive at end of conversation
+         │   Smart Sentence Splitting      │  ← Protect abbreviations/numbers/quotes
+         │   Attention Tracking        │  ← Extract keywords and increase attention score
+         │   ↓                │
+         │   Stored in Redis        │  ← Available for next retrieval
+         └───────────────────┘
+                   │
+         ┌─────────▼─────────┐
+         │   [cron] Every 2h     │  ← Background maintenance
+         │   ① Multi-level Consolidation       │  ← Same topic→LLM summarization→level+1
+         │   ② Selective Forgetting     │  ← Cleanup low-value fragments
+         └───────────────────┘
+```
+
+## License
+
+MIT

fragmented_memory-1.0.0/README.md

@@ -0,0 +1,224 @@
+# Fragmented Memory Plugin for Hermes Agent
+
+The Fragmented Memory system automatically retrieves relevant memory fragments and injects them into the conversation context for each dialogue.
+
+```text
+User: "How did we set up that React project structure last time?"
+                      ↓
+         Fragmented Memory System     ← Redis + RediSearch
+                      ↓
+    ┌─────────────────────────────────────┐
+    │  [1] User prefers TypeScript + Vite    │
+    │  [2] Previous projects used pinia state management │
+    │  [3] Backend suggested using .NET 10 implementation       │
+    └─────────────────────────────────────┘
+                      ↓
+         Model directly uses fragments to answer
+```
+
+## Features
+
+- **Semantic Splitting** — auto-split conversations into standalone fragments
+- **BM25 Full-Text Search** — works out of the box with no external API
+- **Optional Vector Search** — KNN via RediSearch (OpenAI / DashScope embedder)
+- **Time Decay** — newer fragments rank higher (60-day half-life configurable)
+- **Sentiment Weighting** — emotional fragments get priority
+- **User Feedback** — mark fragments useful/useless to improve ranking
+- **Hot Topic Boost** — frequently discussed topics rank higher
+- **Auto Sync** — every turn is archived automatically, memory tool writes are synced
+- **Auto Maintenance** — consolidation + selective forgetting to keep storage tidy
+
+## Requirements
+
+- **Python 3.10+**
+- **Hermes Agent 0.12+** — provides `MemoryProvider` interface
+- **Redis 7+** — with RediSearch module (v2.6+)
+- **jieba** — Chinese tokenization (auto-installed)
+- **Embedding API** (optional) — OpenAI / DashScope / any compatible `/v1/embeddings` service
+
+## Installation
+
+```bash
+pip install fragmented-memory
+```
+
+Or install directly from GitHub:
+
+```bash
+pip install git+https://github.com/j-zly/fragmented-memory.git
+```
+
+## Configuration
+
+Configuration precedence (high to low): **Environment variables > JSON config file > config.yaml inline > defaults**
+
+### 1. Create Redis Index (first-time usage)
+
+The code will auto-create (`ensure_index()`), or execute manually:
+
+```bash
+redis-cli FT.CREATE idx:memories ON HASH PREFIX 1 "memory:frag:" SCHEMA \
+    content TEXT WEIGHT 1 \
+    tags TAG SEPARATOR "," \
+    category TAG SEPARATOR "," \
+    source TEXT WEIGHT 1 \
+    created TEXT WEIGHT 0 \
+    fragment_type TAG SEPARATOR "," \
+    embed_bin VECTOR FLAT 6 TYPE FLOAT32 DIM 1536 DISTANCE_METRIC COSINE
+```
+
+> Dimension (DIM) is dynamically adjusted based on the embedding model used, default 1536.
+> For Docker: `docker run -d --name redis-stack -p 6379:6379 redis/redis-stack:latest`
+
+### 2. Hermes Configuration
+
+Enable in `~/.hermes/config.yaml`:
+
+```yaml
+memory:
+  provider: fragmented
+```
+
+Detailed configuration is recommended in a JSON config file (not embedded in config.yaml):
+
+`~/.config/fragmented-memory/config.json`:
+
+```json
+{
+  "redis_host": "127.0.0.1",
+  "redis_port": 6379,
+  "top_k": 5,
+  "candidate_k": 10,
+  "embedder": {
+    "provider": "dashscope",
+    "api_key": "sk-xxx",
+    "model": "text-embedding-v2"
+  }
+}
+```
+
+If `embedder` is not configured, only BM25 full-text search mode will be used.
+
+Also supports environment variable configuration (highest precedence):
+
+```bash
+export FRAGMENTED_REDIS_HOST=127.0.0.1
+export FRAGMENTED_REDIS_PORT=6379
+export FRAGMENTED_TOP_K=5
+export FRAGMENTED_EMBEDDER=dashscope
+export FRAGMENTED_EMBEDDER_MODEL=text-embedding-v2
+export OPENAI_API_KEY=sk-xxx        # embedder API key
+```
+
+### 3. Restart Gateway
+
+```bash
+# For CLI mode, restart session is sufficient
+# For Gateway mode, restart the process
+```
+
+## Configuration Reference
+
+| Config Item | Environment Variable | Default Value | Description |
+|-------------|---------------------|---------------|-------------|
+| `redis_host` | `FRAGMENTED_REDIS_HOST` | `127.0.0.1` | Redis address |
+| `redis_port` | `FRAGMENTED_REDIS_PORT` | `6379` | Redis port |
+| `top_k` | `FRAGMENTED_TOP_K` | `5` | Number of final fragments returned |
+| `candidate_k` | `FRAGMENTED_CANDIDATE_K` | `10` | Candidate fragments count (for KNN) |
+| `tag_filter` | `FRAGMENTED_TAG_FILTER` | `""` | Tag filtering (comma-separated) |
+| `bm25_limit` | `FRAGMENTED_BM25_LIMIT` | `10` | BM25 search candidate count |
+| `decay_half_days` | `FRAGMENTED_DECAY_HALF_DAYS` | `60` | Time decay half-life (days) |
+| `embed_cache_ttl` | `FRAGMENTED_EMBED_CACHE_TTL` | `3600` | Embedding cache TTL (seconds) |
+| `sentiment_boost_positive` | — | `1.5` | Positive fragment weight multiplier |
+| `sentiment_boost_negative` | — | `1.3` | Negative fragment weight multiplier |
+| `feedback_positive_boost` | — | `1.3` | Positive feedback bonus weight |
+| `feedback_negative_penalty` | — | `0.5` | Negative feedback penalty coefficient |
+| `hot_topic_boost` | — | `1.2` | Hot topic weighting multiplier |
+| `embedder.provider` | `FRAGMENTED_EMBEDDER` | `openai` | `openai` / `dashscope` |
+| `embedder.api_key` | `OPENAI_API_KEY` | — | Embedding API key |
+| `embedder.base_url` | `FRAGMENTED_EMBEDDER_URL` | `https://api.openai.com/v1` | API endpoint |
+| `embedder.model` | `FRAGMENTED_EMBEDDER_MODEL` | `text-embedding-3-small` | Embedding model name |
+| `consolidate_min_group` | — | `2` | Minimum fragments to trigger consolidation |
+| `consolidate_max_age_hours` | — | `72` | Minimum age (hours) before fragments can be consolidated |
+| `forget_max_age_days` | — | `30` | Number of days before fragments might be forgotten |
+| `forget_dry_run` | — | `true` | Safe mode for forgetting: only count, don't delete |
+| `hot_topic_decay_half_days` | — | `30` | Hot topic time decay half-life (days) |
+| `emotion_intensity_factor` | — | `0.4` | Emotion intensity → weight coefficient (0=disabled, 1=max) |
+| `attention_boost_max` | — | `1.5` | Maximum attention weighting value |
+| `attention_base_increment` | — | `2.0` | Base attention increment per mention |
+| `attention_emotion_factor` | — | `1.5` | Emotion intensity amplification factor for attention |
+
+> `sentiment_*`, `feedback_*`, `hot_topic_*` and other ranking weight parameters currently only support configuration through JSON config file, not environment variables. Set to `1.0` to disable the effect of that dimension.
+
+### Embedding Models and Dimensions
+
+| Model | Dimensions |
+|-------|------------|
+| OpenAI text-embedding-3-small | 1536 |
+| OpenAI text-embedding-3-large | 3072 |
+| OpenAI text-embedding-ada-002 | 1536 |
+| DashScope text-embedding-v2 | 1536 |
+| DashScope text-embedding-v3 | 1024 |
+
+Dimensions are automatically detected, switching models doesn't require reconfiguration.
+
+### Synonym Table
+
+Stored in Redis Hash `fragmented:synonyms`, expanded at search time to improve recall:
+
+```bash
+redis-cli HSET fragmented:synonyms setup '["install","configure","deploy","setup"]'
+redis-cli HSET fragmented:synonyms fix '["fix","modify","correct","repair","solve"]'
+```
+
+## Verification
+
+Check logs after startup:
+
+```
+Memory provider 'fragmented' registered (0 tools)
+fragmented: connected (session=xxx, top_k=5, tag_filter=(none))
+fragmented: BM25-only mode (no embedder configured)
+```
+
+## Architecture
+
+```
+┌────────────────────────────────────────────────────────┐
+│                    User sends message                  │
+└──────────────────┬─────────────────────────────────────┘
+                   │
+         ┌─────────▼─────────┐
+         │   prefetch()       │  ← Automatically triggered
+         │   ↓                │
+         │  BM25 Full-Text Search │  ← Default, zero cost
+         │  (KNN Vector search) │  ← Optional (needs embedder)
+         │   ↓                │
+         │  Six-dimensional Re-ranking │  ← Similarity × Time decay
+         │                    │    × Emotion × Feedback × Hot Topic × Attention
+         │   ↓                │
+         │  Top N Injected into Context   │
+         └─────────┬─────────┘
+                   │
+         ┌─────────▼─────────┐
+         │   Model Response   │  ← Fragments can be used as reference
+         └───────────────────┘
+                   │
+         ┌─────────▼─────────┐
+         │   sync_turn()      │  ← Automatically archive at end of conversation
+         │   Smart Sentence Splitting      │  ← Protect abbreviations/numbers/quotes
+         │   Attention Tracking        │  ← Extract keywords and increase attention score
+         │   ↓                │
+         │   Stored in Redis        │  ← Available for next retrieval
+         └───────────────────┘
+                   │
+         ┌─────────▼─────────┐
+         │   [cron] Every 2h     │  ← Background maintenance
+         │   ① Multi-level Consolidation       │  ← Same topic→LLM summarization→level+1
+         │   ② Selective Forgetting     │  ← Cleanup low-value fragments
+         └───────────────────┘
+```
+
+## License
+
+MIT

fragmented_memory-1.0.0/pyproject.toml

@@ -0,0 +1,34 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "fragmented-memory"
+version = "1.0.0"
+description = "Fragmented memory system for Hermes Agent — automatically retrieve relevant memory fragments and inject them into conversation context"
+readme = "README.md"
+license = {text = "MIT"}
+requires-python = ">=3.10"
+authors = [
+    {name = "j-zly"},
+]
+keywords = ["hermes-agent", "memory", "rag", "redisearch", "llm"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+
+dependencies = [
+    "redis>=5.0",
+    "jieba>=0.42",
+]
+
+[tool.setuptools.packages.find]
+where = ["src"]

fragmented_memory-1.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+