llmemory 0.1.13 → 0.1.15

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a188251e04aac90f929fcc952f7902a8e9b86728742ab88e5f209953821efd30
-  data.tar.gz: 01f3cd9d68c50a1a52ed33683b973a379073890db1ae5a3073676fd598581eca
+  metadata.gz: 723fae20d0310ccaeaf9ba600148061d17b2a0b29f933d455d1cf656dee85636
+  data.tar.gz: a135ea1661af46e96843bf52744e8004d0ebe7e8d94b0c46a097c36df53d5bc4
 SHA512:
-  metadata.gz: e24d411ffaca985dc360bc6a6d271f8fd57c687cef26c8242e047fab6bcda400b3764b78aeea8d58ec253bc33ce48b89a46d6a8bdb71db8152c4d175880f3b87
-  data.tar.gz: d7097e4c6cc8442088f5adb0614015b8d9d0541db8f4843e729c8e94cfefe0faf49969e9abb75381b070386c53623631b937fc72989a1f7cb0aa27b3b45a9171
+  metadata.gz: 256caaee94233d5e57b8d9e6007fe1ced57d35e21d40260ce34b2803ba0ef3593b66668aa06334e647edd103aa431113e38b639776163d71153c4b9bac68c1a1
+  data.tar.gz: 33cd1726e9f7bb3328610bafabca5ebfe51f080e7d34c523fc0b363eb290b353c9109937f2782ed7e60906965236e79229838e32c698d2f0e2f73aa2d421970b

data/README.md

@@ -2,6 +2,8 @@
 
 Persistent memory system for LLM agents. Implements short-term checkpointing, long-term memory (file-based or **graph-based**), retrieval with time decay, and maintenance jobs. You can inspect memory from the **CLI** or, in Rails apps, from an optional **dashboard**.
 
+Includes advanced memory management features inspired by [OpenClaw](https://github.com/openclaw/openclaw): pre-compaction memory flush, hybrid search (BM25 + vector), tool result pruning, context window tracking, session lifecycle management, daily memory logs, and auto-recall.
+
 ## Installation
 
 Add to your Gemfile:
@@ -40,11 +42,14 @@ memory.compact!(max_bytes: 8192)  # or use config default
 memory.clear_session!
 ```
 
-- **`add_message(role:, content:)`** — Persists messages in short-term.
+- **`add_message(role:, content:)`** — Persists messages in short-term. Supports `user`, `assistant`, `system`, `tool`, and `tool_result` roles.
 - **`messages`** — Returns the current conversation history.
 - **`retrieve(query, max_tokens: nil)`** — Returns combined context: recent conversation + relevant long-term memories.
+- **`recall_for(query: nil)`** — Auto-recall: returns context for the given query (or last user message if `query` is nil). Only active when `auto_recall_enabled` is true.
 - **`consolidate!`** — Extracts facts from the current conversation and stores them in long-term.
-- **`compact!(max_bytes: nil)`** — Compacts short-term memory by summarizing old messages when byte size exceeds limit. Uses LLM to create a summary, keeping recent messages intact.
+- **`compact!(max_bytes: nil)`** — Compacts short-term memory by summarizing old messages when byte size exceeds limit. Automatically flushes to long-term before compacting when over `memory_flush_threshold_tokens`.
+- **`prune!(mode: nil)`** — Prunes oversized tool results (soft-trim or hard-clear). Only when `prune_tool_results_enabled` is true.
+- **`check_context_window!`** — Triggers consolidate and compact when context exceeds configured thresholds.
 - **`clear_session!`** — Clears short-term only.
 
 ## Configuration
@@ -64,6 +69,37 @@ Llmemory.configure do |config|
   config.max_retrieval_tokens = 2000
   config.prune_after_days = 90
   config.compact_max_bytes = 8192  # max bytes before compact! triggers
+
+  # Pre-compaction memory flush (prevents knowledge loss when compacting)
+  config.memory_flush_enabled = true
+  config.memory_flush_threshold_tokens = 4000
+
+  # Hybrid search (BM25 + vector) and MMR re-ranking
+  config.hybrid_search_enabled = true
+  config.bm25_weight = 0.3
+  config.mmr_enabled = false
+  config.mmr_lambda = 0.7
+
+  # Tool result pruning (soft-trim or hard-clear for tool/tool_result messages)
+  config.prune_tool_results_enabled = false
+  config.prune_tool_results_mode = :soft_trim
+  config.prune_tool_results_max_bytes = 2048
+
+  # Context window tracking and auto-consolidation
+  config.context_window_tokens = 128_000
+  config.reserve_tokens = 16_384
+  config.keep_recent_tokens = 20_000
+
+  # Session lifecycle management
+  config.session_idle_minutes = 60
+  config.session_prune_after_days = 30
+  config.session_max_entries_per_user = 500
+
+  # Daily memory logs (file-based, FileStorage only)
+  config.daily_logs_enabled = false
+
+  # Auto-recall (inject relevant memories before each LLM turn)
+  config.auto_recall_enabled = false
 end
 ```
 
@@ -159,6 +195,71 @@ candidates = memory.search_candidates("job", top_k: 20)
 
 **Graph storage:** `:memory` (in-memory) or `:active_record` (Rails). For ActiveRecord, run `rails g llmemory:install` and migrate; the migration creates `llmemory_nodes`, `llmemory_edges`, and `llmemory_embeddings` (pgvector). Enable the `vector` extension in PostgreSQL for embeddings.
 
+## Advanced Memory Management
+
+These features improve robustness and efficiency, inspired by OpenClaw's memory system.
+
+### Pre-Compaction Memory Flush
+
+Before compacting short-term memory, llmemory can automatically consolidate the conversation into long-term storage. This prevents knowledge loss when the context is summarized.
+
+- **`memory_flush_enabled`** — When true, `compact!` calls `consolidate!` first when messages exceed `memory_flush_threshold_tokens`.
+- **`maybe_flush_memory!`** — Call explicitly to flush when approaching context limits.
+
+### Hybrid Search (BM25 + Vector)
+
+Retrieval combines keyword matching (BM25) with vector similarity for more robust search. Optional MMR (Maximal Marginal Relevance) re-ranking improves result diversity.
+
+- **`hybrid_search_enabled`** — Combines BM25 and vector scores.
+- **`bm25_weight`** — Weight for BM25 (0–1); remainder is vector score.
+- **`mmr_enabled`** — Re-ranks results for diversity.
+- **`mmr_lambda`** — Balance between relevance and diversity (0–1).
+
+### Tool Result Pruning
+
+Large tool outputs can consume most of the context window. Pruning selectively trims `tool` and `tool_result` messages while keeping user/assistant intact.
+
+- **`prune_tool_results_enabled`** — When true, `retrieve` uses pruned messages and `prune!` is available.
+- **`prune_tool_results_mode`** — `:soft_trim` (keep head+tail) or `:hard_clear` (replace with placeholder).
+- **`prune_tool_results_max_bytes`** — Max bytes before soft-trim applies.
+
+### Context Window Tracking
+
+Track estimated tokens and trigger consolidation/compaction automatically.
+
+- **`context_tokens`** — Returns estimated token count for current messages.
+- **`should_auto_consolidate?`** — True when over `context_window_tokens - reserve_tokens`.
+- **`check_context_window!`** — Runs consolidate and compact when thresholds are exceeded.
+
+### Session Lifecycle Management
+
+Clean up stale or idle sessions to control storage usage.
+
+```ruby
+lifecycle = Llmemory::ShortTerm::SessionLifecycle.new
+lifecycle.cleanup_idle_sessions!(user_id: "user_123", idle_minutes: 60)
+lifecycle.cleanup_stale_sessions!(user_id: "user_123", prune_after_days: 30)
+lifecycle.enforce_max_entries!(user_id: "user_123", max_entries: 500)
+```
+
+Sessions store `last_activity_at` automatically on each save.
+
+### Daily Memory Logs
+
+With `daily_logs_enabled` and FileStorage, file-based memory writes to `memory/YYYY-MM-DD.md` per user. Today's and yesterday's logs are included in retrieval. Useful for temporal organization and human-readable logs.
+
+### Auto-Recall
+
+When `auto_recall_enabled` is true, call `recall_for(query: nil)` before each LLM turn. If `query` is nil, the last user message is used as the search query. Returns combined context without explicit `retrieve` calls.
+
+```ruby
+Llmemory.configure { |c| c.auto_recall_enabled = true }
+# Before each LLM call:
+context = memory.recall_for(query: user_message)
+# Or use last user message automatically:
+context = memory.recall_for
+```
+
 ## Lower-Level APIs
 
 ### Short-Term Memory (Checkpointing)
@@ -333,7 +434,7 @@ MCP_TOKEN=your-secret-token llmemory mcp serve --http --port 443 \
 | `memory_retrieve` | Get context optimized for LLM inference (supports timeline context) |
 | `memory_timeline` | Get chronological timeline of recent memories |
 | `memory_timeline_context` | Get N items before/after a specific memory |
-| `memory_add_message` | Add message to short-term conversation |
+| `memory_add_message` | Add message to short-term conversation (roles: user, assistant, system, tool, tool_result) |
 | `memory_consolidate` | Extract facts from conversation to long-term |
 | `memory_stats` | Get memory statistics for a user |
 | `memory_info` | Documentation on how to use the tools |

data/lib/generators/llmemory/install/templates/create_llmemory_tables.rb

@@ -16,6 +16,7 @@ class CreateLlmemoryTables < ActiveRecord::Migration[7.0]
       t.string :category, null: false
       t.text :content, null: false
       t.string :source_resource_id
+      t.float :importance, default: 0.7
       t.timestamps
     end
     add_index :llmemory_items, :user_id

data/lib/llmemory/configuration.rb

@@ -16,7 +16,32 @@ module Llmemory
                   :time_decay_half_life_days,
                   :max_retrieval_tokens,
                   :prune_after_days,
-                  :compact_max_bytes
+                  :compact_max_bytes,
+                  :memory_flush_enabled,
+                  :memory_flush_threshold_tokens,
+                  :hybrid_search_enabled,
+                  :bm25_weight,
+                  :mmr_enabled,
+                  :mmr_lambda,
+                  :prune_tool_results_enabled,
+                  :prune_tool_results_mode,
+                  :prune_tool_results_max_bytes,
+                  :context_window_tokens,
+                  :reserve_tokens,
+                  :keep_recent_tokens,
+                  :session_idle_minutes,
+                  :session_prune_after_days,
+                  :session_max_entries_per_user,
+                  :daily_logs_enabled,
+                  :auto_recall_enabled,
+                  :noise_filter_enabled,
+                  :noise_filter_min_chars,
+                  :flush_once_per_cycle_seconds,
+                  :overflow_recovery_enabled,
+                  :embedding_cache_enabled,
+                  :embedding_cache_max_entries,
+                  :max_message_chars,
+                  :message_sanitizer_enabled
 
     def initialize
       @llm_provider = :openai
@@ -34,6 +59,31 @@ module Llmemory
       @max_retrieval_tokens = 2000
       @prune_after_days = 90
       @compact_max_bytes = 8192
+      @memory_flush_enabled = true
+      @memory_flush_threshold_tokens = 4000
+      @hybrid_search_enabled = true
+      @bm25_weight = 0.3
+      @mmr_enabled = false
+      @mmr_lambda = 0.7
+      @prune_tool_results_enabled = false
+      @prune_tool_results_mode = :soft_trim
+      @prune_tool_results_max_bytes = 2048
+      @context_window_tokens = 128_000
+      @reserve_tokens = 16_384
+      @keep_recent_tokens = 20_000
+      @session_idle_minutes = 60
+      @session_prune_after_days = 30
+      @session_max_entries_per_user = 500
+      @daily_logs_enabled = false
+      @auto_recall_enabled = false
+      @noise_filter_enabled = false
+      @noise_filter_min_chars = 10
+      @flush_once_per_cycle_seconds = 60
+      @overflow_recovery_enabled = false
+      @embedding_cache_enabled = true
+      @embedding_cache_max_entries = 10_000
+      @max_message_chars = 32_000
+      @message_sanitizer_enabled = false
     end
   end
 

data/lib/llmemory/extractors/fact_extractor.rb

@@ -14,7 +14,9 @@ module Llmemory
           Extract discrete facts from this conversation.
           Focus on preferences, behaviors, and important details.
           Conversation: #{conversation_text}
-          Return as JSON array of objects with "content" key. Example: [{"content": "User prefers Ruby"}, {"content": "User is vegan"}]
+          Return as JSON array of objects with "content" and "importance" (0-1) keys.
+          Importance: 0.8-0.95 for preferences/corrections/decisions, 0.5-0.8 for factual context, 0.3-0.5 for ephemeral.
+          Example: [{"content": "User prefers Ruby", "importance": 0.9}, {"content": "User mentioned the weather", "importance": 0.4}]
         PROMPT
         response = @llm.invoke(prompt.strip)
         parse_items_response(response)
@@ -56,7 +58,12 @@ module Llmemory
       def parse_items_response(response)
         json = extract_json_array(response)
         return [] unless json
-        json.map { |item| item.is_a?(Hash) ? item : { "content" => item.to_s } }
+        json.map do |item|
+          h = item.is_a?(Hash) ? item : { "content" => item.to_s }
+          imp = h["importance"] || h[:importance]
+          h["importance"] = imp.nil? ? 0.7 : (imp.to_f.between?(0, 1) ? imp.to_f : 0.7)
+          h
+        end
       end
 
       def extract_json_array(response)

data/lib/llmemory/long_term/file_based/memory.rb

@@ -4,6 +4,7 @@ require_relative "resource"
 require_relative "item"
 require_relative "category"
 require_relative "storage"
+require_relative "../../noise_filter"
 
 module Llmemory
   module LongTerm
@@ -17,16 +18,21 @@ module Llmemory
         end
 
         def memorize(conversation_text)
-          resource_id = save_resource(conversation_text)
-          items = @extractor.extract_items(conversation_text)
+          text = Llmemory.configuration.noise_filter_enabled ? NoiseFilter.filter?(conversation_text) : conversation_text.to_s
+          return true if text.strip.empty?
+
+          resource_id = save_resource(text)
+          append_to_daily_log(text) if Llmemory.configuration.daily_logs_enabled && @storage.respond_to?(:save_daily_log_entry)
+          items = @extractor.extract_items(text)
           updates_by_category = {}
 
           items.each do |item|
             content = item.is_a?(Hash) ? (item["content"] || item[:content]) : item.to_s
+            importance = (item["importance"] || item[:importance] || 0.7).to_f
             cat = @extractor.classify_item(content)
             updates_by_category[cat] ||= []
             updates_by_category[cat] << content.to_s
-            save_item(category: cat, item: item, source_resource_id: resource_id)
+            save_item(category: cat, item: item, source_resource_id: resource_id, importance: importance)
           end
 
           updates_by_category.each do |category, new_memories|
@@ -47,12 +53,20 @@ module Llmemory
           uid = user_id || @user_id
           items = @storage.search_items(uid, query)
           resources = @storage.search_resources(uid, query)
+          daily_logs = load_daily_logs_for_retrieval(uid) if Llmemory.configuration.daily_logs_enabled && @storage.respond_to?(:load_daily_logs)
+          category_summaries = load_category_summaries_as_candidates(uid, query)
           out = []
+
+          category_summaries.each do |c|
+            out << c.merge(evergreen: true)
+          end
+
           items.first(top_k).each do |i|
             out << {
               text: i[:content] || i["content"],
               timestamp: i[:created_at] || i["created_at"],
-              score: 1.0
+              score: (i[:importance] || i["importance"] || 1.0).to_f,
+              evergreen: i[:evergreen] || i["evergreen"]
             }
           end
           resources.first([top_k - out.size, 0].max).each do |r|
@@ -62,6 +76,11 @@ module Llmemory
               score: 0.9
             }
           end
+          if daily_logs
+            daily_logs.each do |log|
+              out << { text: log[:content], timestamp: log[:date].to_time, score: 0.85 }
+            end
+          end
           out
         end
 
@@ -73,9 +92,37 @@ module Llmemory
           @storage.save_resource(@user_id, text)
         end
 
-        def save_item(category:, item:, source_resource_id:)
+        def save_item(category:, item:, source_resource_id:, importance: 0.7)
           content = item.is_a?(Hash) ? item["content"] || item[:content] : item.to_s
-          @storage.save_item(@user_id, category: category, content: content, source_resource_id: source_resource_id)
+          @storage.save_item(@user_id, category: category, content: content, source_resource_id: source_resource_id, importance: importance)
+        end
+
+        def append_to_daily_log(conversation_text)
+          summary = conversation_text.length > 500 ? "#{conversation_text[0..500]}..." : conversation_text
+          @storage.save_daily_log_entry(@user_id, Date.today, summary)
+        end
+
+        def load_daily_logs_for_retrieval(user_id)
+          today = Date.today
+          yesterday = today - 1
+          logs = @storage.load_daily_logs(user_id, from_date: yesterday, to_date: today)
+          logs.map { |l| { date: l[:date], content: "[#{l[:date]}] #{l[:content]}" } }
+        end
+
+        def load_category_summaries_as_candidates(user_id, query)
+          return [] unless @storage.respond_to?(:list_categories)
+
+          categories = @storage.list_categories(user_id)
+          return [] if categories.empty?
+
+          query_lower = query.to_s.downcase
+          categories.filter_map do |cat|
+            summary = @storage.load_category(user_id, cat)
+            next if summary.to_s.strip.empty?
+            next unless summary.to_s.downcase.include?(query_lower)
+
+            { text: "[#{cat}] #{summary}", timestamp: Time.now, score: 0.95 }
+          end
         end
       end
     end

data/lib/llmemory/long_term/file_based/storages/active_record_storage.rb

@@ -30,16 +30,18 @@ module Llmemory
             id
           end
 
-          def save_item(user_id, category:, content:, source_resource_id:)
+          def save_item(user_id, category:, content:, source_resource_id:, importance: 0.7)
             id = "item_#{SecureRandom.hex(8)}"
-            LlmemoryItem.create!(
+            attrs = {
               id: id,
               user_id: user_id,
               category: category,
               content: content,
               source_resource_id: source_resource_id,
               created_at: Time.current
-            )
+            }
+            attrs[:importance] = importance if LlmemoryItem.column_names.include?("importance")
+            LlmemoryItem.create!(attrs)
             id
           end
 
@@ -96,14 +98,16 @@ module Llmemory
           def replace_items(user_id, ids_to_remove, merged_item)
             LlmemoryItem.where(user_id: user_id, id: ids_to_remove).destroy_all
             created_at = merged_item[:created_at] || Time.current
-            LlmemoryItem.create!(
+            attrs = {
               id: "item_#{SecureRandom.hex(8)}",
               user_id: user_id,
               category: merged_item[:category],
               content: merged_item[:content],
               source_resource_id: merged_item[:source_resource_id],
               created_at: created_at
-            )
+            }
+            attrs[:importance] = merged_item[:importance] if LlmemoryItem.column_names.include?("importance") && merged_item[:importance]
+            LlmemoryItem.create!(attrs)
           end
 
           def archive_items(user_id, item_ids)
@@ -177,13 +181,15 @@ module Llmemory
           end
 
           def row_to_item(r)
-            {
+            h = {
               id: r.id,
               category: r.category,
               content: r.content,
               source_resource_id: r.source_resource_id,
               created_at: r.created_at
             }
+            h[:importance] = r.respond_to?(:importance) ? (r.importance || 0.7).to_f : 0.7
+            h
           end
 
           def row_to_resource(r)

data/lib/llmemory/long_term/file_based/storages/base.rb

@@ -9,7 +9,7 @@ module Llmemory
             raise NotImplementedError, "#{self.class}#save_resource must be implemented"
           end
 
-          def save_item(user_id, category:, content:, source_resource_id:)
+          def save_item(user_id, category:, content:, source_resource_id:, importance: 0.7)
             raise NotImplementedError, "#{self.class}#save_item must be implemented"
           end
 

data/lib/llmemory/long_term/file_based/storages/database_storage.rb

@@ -24,12 +24,12 @@ module Llmemory
             id
           end
 
-          def save_item(user_id, category:, content:, source_resource_id:)
+          def save_item(user_id, category:, content:, source_resource_id:, importance: 0.7)
             ensure_tables!
             id = "item_#{SecureRandom.hex(8)}"
             conn.exec_params(
-              "INSERT INTO llmemory_items (id, user_id, category, content, source_resource_id, created_at) VALUES ($1, $2, $3, $4, $5, $6)",
-              [id, user_id, category, content, source_resource_id, Time.now.utc.iso8601]
+              "INSERT INTO llmemory_items (id, user_id, category, content, source_resource_id, importance, created_at) VALUES ($1, $2, $3, $4, $5, $6, $7)",
+              [id, user_id, category, content, source_resource_id, importance.to_f, Time.now.utc.iso8601]
             )
             id
           end
@@ -67,7 +67,7 @@ module Llmemory
             ensure_tables!
             pattern = "%#{conn.escape_string(query.to_s.downcase)}%"
             rows = conn.exec_params(
-              "SELECT id, category, content, source_resource_id, created_at FROM llmemory_items WHERE user_id = $1 AND LOWER(content) LIKE $2",
+              "SELECT id, category, content, source_resource_id, importance, created_at FROM llmemory_items WHERE user_id = $1 AND LOWER(content) LIKE $2",
               [user_id, pattern]
             )
             rows_to_items(rows)
@@ -97,7 +97,7 @@ module Llmemory
             ensure_tables!
             cutoff = (Time.now - (days * 86400)).utc.iso8601
             rows = conn.exec_params(
-              "SELECT id, category, content, source_resource_id, created_at FROM llmemory_items WHERE user_id = $1 AND created_at < $2 ORDER BY created_at",
+              "SELECT id, category, content, source_resource_id, importance, created_at FROM llmemory_items WHERE user_id = $1 AND created_at < $2 ORDER BY created_at",
               [user_id, cutoff]
             )
             rows_to_items(rows)
@@ -106,7 +106,7 @@ module Llmemory
           def get_all_items(user_id)
             ensure_tables!
             rows = conn.exec_params(
-              "SELECT id, category, content, source_resource_id, created_at FROM llmemory_items WHERE user_id = $1 ORDER BY created_at",
+              "SELECT id, category, content, source_resource_id, importance, created_at FROM llmemory_items WHERE user_id = $1 ORDER BY created_at",
               [user_id]
             )
             rows_to_items(rows)
@@ -125,7 +125,7 @@ module Llmemory
             ensure_tables!
             cutoff = (Time.now - (hours * 3600)).utc.iso8601
             rows = conn.exec_params(
-              "SELECT id, category, content, source_resource_id, created_at FROM llmemory_items WHERE user_id = $1 AND created_at >= $2 ORDER BY created_at",
+              "SELECT id, category, content, source_resource_id, importance, created_at FROM llmemory_items WHERE user_id = $1 AND created_at >= $2 ORDER BY created_at",
               [user_id, cutoff]
             )
             rows_to_items(rows)
@@ -179,7 +179,7 @@ module Llmemory
 
           def list_items(user_id:, category: nil, limit: nil)
             ensure_tables!
-            sql = "SELECT id, category, content, source_resource_id, created_at FROM llmemory_items WHERE user_id = $1"
+            sql = "SELECT id, category, content, source_resource_id, importance, created_at FROM llmemory_items WHERE user_id = $1"
             params = [user_id]
             if category
               sql += " AND category = $2"
@@ -257,10 +257,12 @@ module Llmemory
                 category TEXT NOT NULL,
                 content TEXT NOT NULL,
                 source_resource_id TEXT,
+                importance REAL DEFAULT 0.7,
                 created_at TIMESTAMPTZ NOT NULL
               );
               CREATE INDEX IF NOT EXISTS idx_llmemory_items_user_id ON llmemory_items(user_id);
             SQL
+            conn.exec("ALTER TABLE llmemory_items ADD COLUMN IF NOT EXISTS importance REAL DEFAULT 0.7") rescue nil
             conn.exec(<<~SQL)
               CREATE TABLE IF NOT EXISTS llmemory_categories (
                 user_id TEXT NOT NULL,
@@ -279,6 +281,7 @@ module Llmemory
                 category: r["category"],
                 content: r["content"],
                 source_resource_id: r["source_resource_id"],
+                importance: (r["importance"] || 0.7).to_f,
                 created_at: Time.parse(r["created_at"])
               }
             end

data/lib/llmemory/long_term/file_based/storages/file_storage.rb

@@ -24,7 +24,7 @@ module Llmemory
             id
           end
 
-          def save_item(user_id, category:, content:, source_resource_id:)
+          def save_item(user_id, category:, content:, source_resource_id:, importance: 0.7)
             ensure_user_dir(user_id)
             seq = next_seq(user_id, "item_id_seq")
             id = "item_#{seq}"
@@ -34,6 +34,7 @@ module Llmemory
               category: category,
               content: content,
               source_resource_id: source_resource_id,
+              importance: importance,
               created_at: Time.now.iso8601
             }
             File.write(path, JSON.generate(data))
@@ -124,6 +125,29 @@ module Llmemory
             resource_ids.each { |id| File.delete(resource_path(user_id, id)) if File.file?(resource_path(user_id, id)) }
           end
 
+          def save_daily_log_entry(user_id, date, content)
+            ensure_user_dir(user_id, "memory")
+            path = daily_log_path(user_id, date)
+            existing = File.file?(path) ? File.read(path) : ""
+            entry = "#{Time.now.strftime('%H:%M')} #{content}\n"
+            File.write(path, existing + entry)
+            true
+          end
+
+          def load_daily_logs(user_id, from_date:, to_date:)
+            from_date = Date.parse(from_date.to_s) if from_date.is_a?(String)
+            to_date = Date.parse(to_date.to_s) if to_date.is_a?(String)
+            dir = user_path(user_id, "memory")
+            return [] unless Dir.exist?(dir)
+
+            (from_date..to_date).filter_map do |d|
+              path = daily_log_path(user_id, d)
+              next unless File.file?(path)
+
+              { date: d, content: File.read(path) }
+            end
+          end
+
           def list_users
             return [] unless Dir.exist?(@base_path)
             Dir.children(@base_path).select { |e| File.directory?(File.join(@base_path, e)) && !e.start_with?(".") }
@@ -162,6 +186,11 @@ module Llmemory
             File.join(user_path(user_id, "items"), "#{id}.json")
           end
 
+          def daily_log_path(user_id, date)
+            date_str = date.respond_to?(:strftime) ? date.strftime("%Y-%m-%d") : date.to_s
+            File.join(user_path(user_id, "memory"), "#{date_str}.md")
+          end
+
           def category_path(user_id, category_name)
             safe = category_name.to_s.gsub(%r{[^\w\-.]}, "_")
             File.join(user_path(user_id, "categories"), "#{safe}.md")

data/lib/llmemory/long_term/file_based/storages/memory_storage.rb

@@ -22,7 +22,7 @@ module Llmemory
             id
           end
 
-          def save_item(user_id, category:, content:, source_resource_id:)
+          def save_item(user_id, category:, content:, source_resource_id:, importance: 0.7)
             @item_id_seq += 1
             id = "item_#{@item_id_seq}"
             @items[user_id] << {
@@ -30,6 +30,7 @@ module Llmemory
               category: category,
               content: content,
               source_resource_id: source_resource_id,
+              importance: importance,
               created_at: Time.now
             }
             id

data/lib/llmemory/long_term/graph_based/memory.rb

@@ -5,6 +5,7 @@ require_relative "edge"
 require_relative "knowledge_graph"
 require_relative "conflict_resolver"
 require_relative "storage"
+require_relative "../../noise_filter"
 
 module Llmemory
   module LongTerm
@@ -21,7 +22,10 @@ module Llmemory
         end
 
         def memorize(conversation_text)
-          data = @extractor.extract(conversation_text) rescue { entities: [], relations: [] }
+          text = Llmemory.configuration.noise_filter_enabled ? NoiseFilter.filter?(conversation_text) : conversation_text.to_s
+          return true if text.strip.empty?
+
+          data = @extractor.extract(text) rescue { entities: [], relations: [] }
           data = { entities: [], relations: [] } unless data.is_a?(Hash)
           entities = Array(data[:entities] || data["entities"])
           relations = Array(data[:relations] || data["relations"])

data/lib/llmemory/mcp/tools/memory_add_message.rb

@@ -10,7 +10,7 @@ module Llmemory
           properties: {
             user_id: { type: "string", description: "User identifier" },
             session_id: { type: "string", description: "Session identifier (default: 'default')" },
-            role: { type: "string", enum: ["user", "assistant", "system"], description: "Message role" },
+            role: { type: "string", enum: ["user", "assistant", "system", "tool", "tool_result"], description: "Message role" },
             content: { type: "string", description: "Message content" }
           },
           required: ["user_id", "role", "content"]