mem-llm 1.1.0__tar.gz → 1.2.0__tar.gz

{mem_llm-1.1.0 → mem_llm-1.2.0}/CHANGELOG.md

@@ -5,6 +5,55 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.2.0] - 2025-10-21
+
+### Added
+
+- 📊 **Conversation Summarization**: Automatic conversation history compression
+  - `ConversationSummarizer`: Generates concise summaries from conversation histories
+  - `AutoSummarizer`: Threshold-based automatic summary updates
+  - Token compression: ~40-60% reduction in context size
+  - Key facts extraction: Automatic user profile insights
+  - Configurable thresholds and conversation limits
+  
+- 📤 **Data Export/Import System**: Multi-format and multi-database support
+  - `DataExporter`: Export conversations to JSON, CSV, SQLite, PostgreSQL, MongoDB
+  - `DataImporter`: Import from JSON, CSV, SQLite, PostgreSQL, MongoDB
+  - Auto-create databases: PostgreSQL and MongoDB databases created automatically if missing
+  - Enterprise-ready: Support for analytics (PostgreSQL) and real-time dashboards (MongoDB)
+  - Optional dependencies: `pip install mem-llm[postgresql]`, `pip install mem-llm[mongodb]`, `pip install mem-llm[databases]`
+  
+- 🗄️ **In-Memory Database Support**: Temporary database operations
+  - `db_path=":memory:"` parameter for MemAgent
+  - No file creation: Perfect for testing and temporary workflows
+  - Full SQL functionality without persistent storage
+
+### Changed
+
+- 🔇 **Reduced Logging Verbosity**: Cleaner console output
+  - Default log level changed from INFO to WARNING
+  - Less noise in production environments
+  - Users can still enable detailed logs via config
+  - Examples suppress logs for cleaner demonstrations
+  
+- 📦 **Enhanced Package Structure**: Better optional dependencies
+  - `pip install mem-llm[postgresql]` - PostgreSQL support only
+  - `pip install mem-llm[mongodb]` - MongoDB support only  
+  - `pip install mem-llm[databases]` - Both PostgreSQL and MongoDB
+  - `pip install mem-llm[all]` - Everything included
+
+### Fixed
+
+- 🗄️ **Database Path Handling**: SQLite files now organized in memories/ folder
+  - All SQLite files (.db, .db-shm, .db-wal) now in memories/ directory
+  - Cleaner workspace: No database files cluttering project root
+  - Automatic directory creation: memories/ folder created if missing
+  
+- 🔧 **MemAgent db_path Parameter**: Added missing parameter
+  - New `db_path` parameter in MemAgent.__init__()
+  - Enables custom database locations and in-memory databases
+  - Better control over database file placement
+
 ## [1.1.0] - 2025-10-21
 
 ### Added

{mem_llm-1.1.0/mem_llm.egg-info → mem_llm-1.2.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.2
 Name: mem-llm
-Version: 1.1.0
-Summary: Memory-enabled AI assistant with local LLM support - Now with security and performance improvements
+Version: 1.2.0
+Summary: Memory-enabled AI assistant with local LLM support - Now with data import/export and multi-database support
 Author-email: "C. Emre Karataş" <karatasqemre@gmail.com>
 License: MIT
 Project-URL: Homepage, https://github.com/emredeveloper/Mem-LLM
@@ -33,6 +33,24 @@ Requires-Dist: flask-cors>=4.0.0; extra == "web"
 Provides-Extra: api
 Requires-Dist: fastapi>=0.104.0; extra == "api"
 Requires-Dist: uvicorn>=0.24.0; extra == "api"
+Provides-Extra: postgresql
+Requires-Dist: psycopg2-binary>=2.9.9; extra == "postgresql"
+Provides-Extra: mongodb
+Requires-Dist: pymongo>=4.6.0; extra == "mongodb"
+Provides-Extra: databases
+Requires-Dist: psycopg2-binary>=2.9.9; extra == "databases"
+Requires-Dist: pymongo>=4.6.0; extra == "databases"
+Provides-Extra: all
+Requires-Dist: pytest>=7.4.0; extra == "all"
+Requires-Dist: pytest-cov>=4.1.0; extra == "all"
+Requires-Dist: black>=23.7.0; extra == "all"
+Requires-Dist: flake8>=6.1.0; extra == "all"
+Requires-Dist: flask>=3.0.0; extra == "all"
+Requires-Dist: flask-cors>=4.0.0; extra == "all"
+Requires-Dist: fastapi>=0.104.0; extra == "all"
+Requires-Dist: uvicorn>=0.24.0; extra == "all"
+Requires-Dist: psycopg2-binary>=2.9.9; extra == "all"
+Requires-Dist: pymongo>=4.6.0; extra == "all"
 
 # 🧠 Mem-LLM
 
@@ -44,16 +62,23 @@ Requires-Dist: uvicorn>=0.24.0; extra == "api"
 
 Mem-LLM is a powerful Python library that brings persistent memory capabilities to local Large Language Models. Build AI assistants that remember user interactions, manage knowledge bases, and work completely offline with Ollama.
 
-## 🆕 What's New in v1.1.0
+## 🔗 Links
 
-- 🛡️ **Prompt Injection Protection**: Detects and blocks 15+ attack patterns (opt-in with `enable_security=True`)
-- ⚡ **Thread-Safe Operations**: Fixed all race conditions, supports 200+ concurrent writes
-- 🔄 **Retry Logic**: Exponential backoff for network errors (3 retries: 1s, 2s, 4s)
-- 📝 **Structured Logging**: Production-ready logging with `MemLLMLogger`
-- 💾 **SQLite WAL Mode**: Write-Ahead Logging for better concurrency (15K+ msg/s)
-- ✅ **100% Backward Compatible**: All v1.0.x code works without changes
+- **PyPI**: https://pypi.org/project/mem-llm/
+- **GitHub**: https://github.com/emredeveloper/Mem-LLM
+- **Issues**: https://github.com/emredeveloper/Mem-LLM/issues
+- **Documentation**: See examples/ directory
 
-[See full changelog](CHANGELOG.md#110---2025-10-21)
+## 🆕 What's New in v1.2.0
+
+- � **Conversation Summarization**: Automatic conversation compression (~40-60% token reduction)
+- 📤 **Data Export/Import**: JSON, CSV, SQLite, PostgreSQL, MongoDB support
+- 🗄️ **Multi-Database**: Enterprise-ready PostgreSQL & MongoDB integration
+- �️ **In-Memory DB**: Use `:memory:` for temporary operations
+- � **Cleaner Logs**: Default WARNING level for production-ready output
+- � **Bug Fixes**: Database path handling, organized SQLite files
+
+[See full changelog](CHANGELOG.md#120---2025-10-21)
 
 ## ✨ Key Features
 
@@ -70,15 +95,38 @@ Mem-LLM is a powerful Python library that brings persistent memory capabilities
 - 🛡️ **Prompt Injection Protection** (v1.1.0+) - Advanced security against prompt attacks (opt-in)
 - ⚡ **High Performance** (v1.1.0+) - Thread-safe operations, 15K+ msg/s throughput
 - 🔄 **Retry Logic** (v1.1.0+) - Automatic exponential backoff for network errors
+- 📊 **Conversation Summarization** (v1.2.0+) - Automatic token compression (~40-60% reduction)
+- 📤 **Data Export/Import** (v1.2.0+) - Multi-format support (JSON, CSV, SQLite, PostgreSQL, MongoDB)
 
 ## 🚀 Quick Start
 
 ### Installation
 
+**Basic Installation:**
 ```bash
 pip install mem-llm
 ```
 
+**With Optional Dependencies:**
+```bash
+# PostgreSQL support
+pip install mem-llm[postgresql]
+
+# MongoDB support
+pip install mem-llm[mongodb]
+
+# All database support (PostgreSQL + MongoDB)
+pip install mem-llm[databases]
+
+# All optional features
+pip install mem-llm[all]
+```
+
+**Upgrade:**
+```bash
+pip install -U mem-llm
+```
+
 ### Prerequisites
 
 Install and start [Ollama](https://ollama.ai):
@@ -391,33 +439,8 @@ stats = agent.get_memory_stats()
 - **MemoryTools**: Search, export, statistics
 - **ConfigManager**: YAML configuration
 - **CLI**: Command-line interface
-
-## 🧪 Testing
-
-Run the comprehensive test suite:
-
-```bash
-# Install dev dependencies
-pip install -r requirements-dev.txt
-
-# Run all tests (34+ automated tests)
-cd tests
-python run_all_tests.py
-
-# Run specific test
-python -m pytest test_mem_agent.py -v
-```
-
-### Test Coverage
-- ✅ Core imports and dependencies
-- ✅ CLI functionality
-- ✅ Ollama connection and models
-- ✅ JSON memory operations
-- ✅ SQL memory operations
-- ✅ MemAgent features
-- ✅ Configuration management
-- ✅ Multi-user scenarios
-- ✅ Hallucination detection
+- **ConversationSummarizer**: Token compression (v1.2.0+)
+- **DataExporter/DataImporter**: Multi-database support (v1.2.0+)
 
 ## 📝 Examples
 
@@ -430,53 +453,32 @@ The `examples/` directory contains ready-to-run demonstrations:
 5. **05_knowledge_base.py** - FAQ/support system
 6. **06_cli_demo.py** - Command-line interface examples
 7. **07_document_config.py** - Configuration from documents
+8. **08_conversation_summarization.py** - Token compression with auto-summary (v1.2.0+)
+9. **09_data_export_import.py** - Multi-format export/import demo (v1.2.0+)
+10. **10_database_connection_test.py** - Enterprise PostgreSQL/MongoDB migration (v1.2.0+)
 
-## 🛠️ Development
-
-### Setup Development Environment
-
-```bash
-git clone https://github.com/emredeveloper/Mem-LLM.git
-cd Mem-LLM
-pip install -e .
-pip install -r requirements-dev.txt
-```
-
-### Running Tests
-
-```bash
-pytest tests/ -v --cov=mem_llm
-```
-
-### Building Package
-
-```bash
-python -m build
-twine upload dist/*
-```
-
-## 📋 Requirements
-
-### Core Dependencies
-- Python 3.8+
-- requests>=2.31.0
-- pyyaml>=6.0.1
-- click>=8.1.0
-
-### Optional Dependencies
-- pytest>=7.4.0 (for testing)
-- flask>=3.0.0 (for web interface)
-- fastapi>=0.104.0 (for API server)
+## 📊 Project Status
 
-## 🤝 Contributing
+- **Version**: 1.2.0
+- **Status**: Production Ready
+- **Last Updated**: October 21, 2025
+- **Test Coverage**: 16/16 automated tests (100% success rate)
+- **Performance**: Thread-safe operations, <1ms search latency
+- **Databases**: SQLite, PostgreSQL, MongoDB, In-Memory
 
-Contributions are welcome! Please feel free to submit a Pull Request.
+## 📈 Roadmap
 
-1. Fork the repository
-2. Create your feature branch (`git checkout -b feature/AmazingFeature`)
-3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
-4. Push to the branch (`git push origin feature/AmazingFeature`)
-5. Open a Pull Request
+- [x] ~~Thread-safe operations~~ (v1.1.0)
+- [x] ~~Prompt injection protection~~ (v1.1.0)
+- [x] ~~Structured logging~~ (v1.1.0)
+- [x] ~~Retry logic~~ (v1.1.0)
+- [x] ~~Conversation Summarization~~ (v1.2.0)
+- [x] ~~Multi-Database Export/Import~~ (v1.2.0)
+- [x] ~~In-Memory Database~~ (v1.2.0)
+- [ ] Web UI dashboard
+- [ ] REST API server
+- [ ] Vector database integration
+- [ ] Advanced analytics dashboard
 
 ## 📄 License
 
@@ -494,35 +496,6 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
 - Inspired by the need for privacy-focused AI assistants
 - Thanks to all contributors and users
 
-## 📊 Project Status
-
-- **Version**: 1.1.0
-- **Status**: Production Ready
-- **Last Updated**: October 21, 2025
-- **Performance**: 15,346 msg/s write throughput, <1ms search latency
-- **Thread-Safe**: Supports 200+ concurrent operations
-- **Test Coverage**: 44+ automated tests (100% success rate)
-
-## 🔗 Links
-
-- **PyPI**: https://pypi.org/project/mem-llm/
-- **GitHub**: https://github.com/emredeveloper/Mem-LLM
-- **Issues**: https://github.com/emredeveloper/Mem-LLM/issues
-- **Documentation**: See examples/ directory
-
-## 📈 Roadmap
-
-- [x] ~~Thread-safe operations~~ (v1.1.0)
-- [x] ~~Prompt injection protection~~ (v1.1.0)
-- [x] ~~Structured logging~~ (v1.1.0)
-- [x] ~~Retry logic~~ (v1.1.0)
-- [ ] Web UI dashboard
-- [ ] REST API server
-- [ ] Vector database integration
-- [ ] Multi-language support
-- [ ] Cloud backup options
-- [ ] Advanced analytics
-
 ---
 
 **⭐ If you find this project useful, please give it a star on GitHub!**

{mem_llm-1.1.0 → mem_llm-1.2.0}/README.md

@@ -8,16 +8,23 @@
 
 Mem-LLM is a powerful Python library that brings persistent memory capabilities to local Large Language Models. Build AI assistants that remember user interactions, manage knowledge bases, and work completely offline with Ollama.
 
-## 🆕 What's New in v1.1.0
+## 🔗 Links
 
-- 🛡️ **Prompt Injection Protection**: Detects and blocks 15+ attack patterns (opt-in with `enable_security=True`)
-- ⚡ **Thread-Safe Operations**: Fixed all race conditions, supports 200+ concurrent writes
-- 🔄 **Retry Logic**: Exponential backoff for network errors (3 retries: 1s, 2s, 4s)
-- 📝 **Structured Logging**: Production-ready logging with `MemLLMLogger`
-- 💾 **SQLite WAL Mode**: Write-Ahead Logging for better concurrency (15K+ msg/s)
-- ✅ **100% Backward Compatible**: All v1.0.x code works without changes
+- **PyPI**: https://pypi.org/project/mem-llm/
+- **GitHub**: https://github.com/emredeveloper/Mem-LLM
+- **Issues**: https://github.com/emredeveloper/Mem-LLM/issues
+- **Documentation**: See examples/ directory
 
-[See full changelog](CHANGELOG.md#110---2025-10-21)
+## 🆕 What's New in v1.2.0
+
+- � **Conversation Summarization**: Automatic conversation compression (~40-60% token reduction)
+- 📤 **Data Export/Import**: JSON, CSV, SQLite, PostgreSQL, MongoDB support
+- 🗄️ **Multi-Database**: Enterprise-ready PostgreSQL & MongoDB integration
+- �️ **In-Memory DB**: Use `:memory:` for temporary operations
+- � **Cleaner Logs**: Default WARNING level for production-ready output
+- � **Bug Fixes**: Database path handling, organized SQLite files
+
+[See full changelog](CHANGELOG.md#120---2025-10-21)
 
 ## ✨ Key Features
 
@@ -34,15 +41,38 @@ Mem-LLM is a powerful Python library that brings persistent memory capabilities
 - 🛡️ **Prompt Injection Protection** (v1.1.0+) - Advanced security against prompt attacks (opt-in)
 - ⚡ **High Performance** (v1.1.0+) - Thread-safe operations, 15K+ msg/s throughput
 - 🔄 **Retry Logic** (v1.1.0+) - Automatic exponential backoff for network errors
+- 📊 **Conversation Summarization** (v1.2.0+) - Automatic token compression (~40-60% reduction)
+- 📤 **Data Export/Import** (v1.2.0+) - Multi-format support (JSON, CSV, SQLite, PostgreSQL, MongoDB)
 
 ## 🚀 Quick Start
 
 ### Installation
 
+**Basic Installation:**
 ```bash
 pip install mem-llm
 ```
 
+**With Optional Dependencies:**
+```bash
+# PostgreSQL support
+pip install mem-llm[postgresql]
+
+# MongoDB support
+pip install mem-llm[mongodb]
+
+# All database support (PostgreSQL + MongoDB)
+pip install mem-llm[databases]
+
+# All optional features
+pip install mem-llm[all]
+```
+
+**Upgrade:**
+```bash
+pip install -U mem-llm
+```
+
 ### Prerequisites
 
 Install and start [Ollama](https://ollama.ai):
@@ -355,33 +385,8 @@ stats = agent.get_memory_stats()
 - **MemoryTools**: Search, export, statistics
 - **ConfigManager**: YAML configuration
 - **CLI**: Command-line interface
-
-## 🧪 Testing
-
-Run the comprehensive test suite:
-
-```bash
-# Install dev dependencies
-pip install -r requirements-dev.txt
-
-# Run all tests (34+ automated tests)
-cd tests
-python run_all_tests.py
-
-# Run specific test
-python -m pytest test_mem_agent.py -v
-```
-
-### Test Coverage
-- ✅ Core imports and dependencies
-- ✅ CLI functionality
-- ✅ Ollama connection and models
-- ✅ JSON memory operations
-- ✅ SQL memory operations
-- ✅ MemAgent features
-- ✅ Configuration management
-- ✅ Multi-user scenarios
-- ✅ Hallucination detection
+- **ConversationSummarizer**: Token compression (v1.2.0+)
+- **DataExporter/DataImporter**: Multi-database support (v1.2.0+)
 
 ## 📝 Examples
 
@@ -394,53 +399,32 @@ The `examples/` directory contains ready-to-run demonstrations:
 5. **05_knowledge_base.py** - FAQ/support system
 6. **06_cli_demo.py** - Command-line interface examples
 7. **07_document_config.py** - Configuration from documents
+8. **08_conversation_summarization.py** - Token compression with auto-summary (v1.2.0+)
+9. **09_data_export_import.py** - Multi-format export/import demo (v1.2.0+)
+10. **10_database_connection_test.py** - Enterprise PostgreSQL/MongoDB migration (v1.2.0+)
 
-## 🛠️ Development
-
-### Setup Development Environment
-
-```bash
-git clone https://github.com/emredeveloper/Mem-LLM.git
-cd Mem-LLM
-pip install -e .
-pip install -r requirements-dev.txt
-```
-
-### Running Tests
-
-```bash
-pytest tests/ -v --cov=mem_llm
-```
-
-### Building Package
-
-```bash
-python -m build
-twine upload dist/*
-```
-
-## 📋 Requirements
-
-### Core Dependencies
-- Python 3.8+
-- requests>=2.31.0
-- pyyaml>=6.0.1
-- click>=8.1.0
-
-### Optional Dependencies
-- pytest>=7.4.0 (for testing)
-- flask>=3.0.0 (for web interface)
-- fastapi>=0.104.0 (for API server)
+## 📊 Project Status
 
-## 🤝 Contributing
+- **Version**: 1.2.0
+- **Status**: Production Ready
+- **Last Updated**: October 21, 2025
+- **Test Coverage**: 16/16 automated tests (100% success rate)
+- **Performance**: Thread-safe operations, <1ms search latency
+- **Databases**: SQLite, PostgreSQL, MongoDB, In-Memory
 
-Contributions are welcome! Please feel free to submit a Pull Request.
+## 📈 Roadmap
 
-1. Fork the repository
-2. Create your feature branch (`git checkout -b feature/AmazingFeature`)
-3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
-4. Push to the branch (`git push origin feature/AmazingFeature`)
-5. Open a Pull Request
+- [x] ~~Thread-safe operations~~ (v1.1.0)
+- [x] ~~Prompt injection protection~~ (v1.1.0)
+- [x] ~~Structured logging~~ (v1.1.0)
+- [x] ~~Retry logic~~ (v1.1.0)
+- [x] ~~Conversation Summarization~~ (v1.2.0)
+- [x] ~~Multi-Database Export/Import~~ (v1.2.0)
+- [x] ~~In-Memory Database~~ (v1.2.0)
+- [ ] Web UI dashboard
+- [ ] REST API server
+- [ ] Vector database integration
+- [ ] Advanced analytics dashboard
 
 ## 📄 License
 
@@ -458,35 +442,6 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
 - Inspired by the need for privacy-focused AI assistants
 - Thanks to all contributors and users
 
-## 📊 Project Status
-
-- **Version**: 1.1.0
-- **Status**: Production Ready
-- **Last Updated**: October 21, 2025
-- **Performance**: 15,346 msg/s write throughput, <1ms search latency
-- **Thread-Safe**: Supports 200+ concurrent operations
-- **Test Coverage**: 44+ automated tests (100% success rate)
-
-## 🔗 Links
-
-- **PyPI**: https://pypi.org/project/mem-llm/
-- **GitHub**: https://github.com/emredeveloper/Mem-LLM
-- **Issues**: https://github.com/emredeveloper/Mem-LLM/issues
-- **Documentation**: See examples/ directory
-
-## 📈 Roadmap
-
-- [x] ~~Thread-safe operations~~ (v1.1.0)
-- [x] ~~Prompt injection protection~~ (v1.1.0)
-- [x] ~~Structured logging~~ (v1.1.0)
-- [x] ~~Retry logic~~ (v1.1.0)
-- [ ] Web UI dashboard
-- [ ] REST API server
-- [ ] Vector database integration
-- [ ] Multi-language support
-- [ ] Cloud backup options
-- [ ] Advanced analytics
-
 ---
 
 **⭐ If you find this project useful, please give it a star on GitHub!**

{mem_llm-1.1.0 → mem_llm-1.2.0}/mem_llm/__init__.py

@@ -43,7 +43,21 @@ try:
 except ImportError:
     __all_enhanced__ = []
 
-__version__ = "1.1.0"
+# Conversation Summarization (v1.2.0+)
+try:
+    from .conversation_summarizer import ConversationSummarizer, AutoSummarizer
+    __all_summarizer__ = ["ConversationSummarizer", "AutoSummarizer"]
+except ImportError:
+    __all_summarizer__ = []
+
+# Data Export/Import (v1.2.0+)
+try:
+    from .data_export_import import DataExporter, DataImporter
+    __all_export_import__ = ["DataExporter", "DataImporter"]
+except ImportError:
+    __all_export_import__ = []
+
+__version__ = "1.2.0"
 __author__ = "C. Emre Karataş"
 
 # CLI
@@ -57,4 +71,4 @@ __all__ = [
     "MemAgent",
     "MemoryManager", 
     "OllamaClient",
-] + __all_tools__ + __all_pro__ + __all_cli__ + __all_security__ + __all_enhanced__
+] + __all_tools__ + __all_pro__ + __all_cli__ + __all_security__ + __all_enhanced__ + __all_summarizer__ + __all_export_import__

{mem_llm-1.1.0 → mem_llm-1.2.0}/mem_llm/config_manager.py

@@ -43,7 +43,7 @@ class ConfigManager:
             "memory": {
                 "backend": "sql",
                 "json_dir": "memories",
-                "db_path": "memories.db",
+                "db_path": "memories/memories.db",
                 "max_conversations_per_user": 1000,
                 "auto_cleanup": True,
                 "cleanup_after_days": 90