mem-llm 1.0.7__tar.gz → 1.0.11__tar.gz

mem_llm-1.0.11/CHANGELOG.md

@@ -0,0 +1,219 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.0.11] - 2025-10-20
+
+### Changed
+- 📝 **Enhanced README.md**: Comprehensive PyPI documentation
+  - Professional badges (version, Python support, license)
+  - Detailed feature list with emojis
+  - Quick start guide with 5-line example
+  - Multiple usage examples (multi-user, advanced config, memory tools)
+  - CLI command documentation
+  - Configuration guide
+  - Complete model compatibility information
+  - Architecture overview
+  - Test coverage details
+  - Development and contribution guidelines
+  - SEO-optimized for PyPI discovery
+
+### Improved
+- 📚 **Documentation Quality**: Better structured for PyPI users
+- 🎯 **User Onboarding**: Clearer getting started instructions
+- 🔍 **Discoverability**: Enhanced keywords and descriptions
+
+## [1.0.10] - 2025-10-20
+
+### Added
+- 🧠 **Dynamic Prompt System**: Context-aware system prompts that adapt to active features
+  - Prevents hallucinations by only including instructions for enabled features
+  - Separate prompt sections for KB, tools, business/personal modes
+  - Automatic feature detection (Knowledge Base presence, tools availability)
+  - Logging shows active features: "✅ Knowledge Base | ❌ Tools | 💾 Memory: SQL"
+- 🔄 **Universal Ollama Model Compatibility**: Full support for ALL Ollama models
+  - Thinking-enabled models (Qwen3, DeepSeek, etc.) now work correctly
+  - Auto-detection and handling of thinking mode
+  - `enable_thinking: false` parameter for direct responses
+  - Fallback extraction from thinking process when needed
+  - Empty response retry with simpler prompts
+- 📊 **Comprehensive Test Suite**: Pre-publish validation system
+  - 34 automated tests covering all major features
+  - Tests: imports, CLI, Ollama, JSON/SQL memory, MemAgent, config, multi-user
+  - User scenario testing with output analysis
+  - Hallucination detection and context verification
+
+### Changed
+- ⚡ **LLM Token Limits**: Increased from 150 to 2000 tokens for thinking models
+- 🧹 **Removed Obsolete Module**: Deleted `prompt_templates.py` (replaced by dynamic system)
+- 📝 **Context Window**: Increased from 2048 to 4096 tokens for better context
+- 🎯 **Response Quality**: Better handling of empty responses with automatic retry
+
+### Fixed
+- 🐛 **Thinking Model Issue**: Qwen3 and similar models now respond correctly
+  - Fixed empty responses from thinking-mode models
+  - Proper content extraction from model responses
+  - System prompt instructions to suppress thinking process
+- 🔧 **Stop Sequences**: Removed problematic stop sequences that interfered with models
+- ⚠️ **Empty Response Handling**: Automatic retry with fallback for reliability
+
+### Improved
+- 🎨 **Prompt Quality**: Feature-specific instructions prevent confusion
+- 🚀 **Model Performance**: Works seamlessly with granite4, qwen3, llama3, and all Ollama models
+- 📈 **User Experience**: No more irrelevant feature mentions in responses
+- 🧪 **Testing Coverage**: Complete validation before releases
+
+### Technical Details
+- Created `mem_llm/dynamic_prompt.py` (350+ lines) - modular prompt builder
+- Modified `mem_llm/mem_agent.py`:
+  - Added `has_knowledge_base` and `has_tools` tracking flags
+  - Implemented `_build_dynamic_system_prompt()` method
+  - Removed ~70 lines of old static prompt code
+  - Added empty response retry logic
+- Modified `mem_llm/llm_client.py`:
+  - Added thinking mode detection and suppression
+  - Increased token limits and context window
+  - Improved response extraction logic
+  - Added fallback for thinking-enabled models
+- Updated `mem_llm/__init__.py` - exported `dynamic_prompt_builder`
+- Cleaned `MANIFEST.in` - removed non-existent files
+- Created `comprehensive_test.py` - 34 automated tests
+- Created `user_test.py` - real-world scenario validation
+
+### Breaking Changes
+- None - fully backward compatible
+
+## [1.0.9] - 2025-10-20
+
+### Added
+- 📝 **PyPI-Optimized README**: Complete rewrite with practical examples
+  - 5 comprehensive usage examples with full code and output
+  - Print statements in all examples for better user experience
+  - Step-by-step workflows showing complete processes
+  - Real-world customer service scenario example
+  - Turkish language support demonstration
+  - User profile extraction example
+- 📄 **Document Configuration Examples**: Added example demonstrating PDF/DOCX/TXT config generation
+- 🧪 **Config Update Testing**: Verification that manual YAML edits work correctly
+
+### Changed
+- 🗑️ **Removed docs folder**: Consolidated documentation into main README
+- 🪵 **Logging Behavior**: Changed from file+console to console-only logging
+  - No more `mem_agent.log` files cluttering workspace
+  - Keeps workspace clean with only `.db` and `.yaml` files
+- 📖 **Example Format**: All examples now include:
+  - Print statements for visibility
+  - Expected output blocks
+  - Full conversation flows
+  - Real usage scenarios
+
+### Fixed
+- 🐛 **Log File Pollution**: Removed FileHandler from logging, only StreamHandler now
+- 📝 **README Examples**: Fixed examples that didn't show actual output or complete process
+
+### Improved
+- 🎯 **User Experience**: Much clearer examples for new users
+- 📚 **Documentation Quality**: Professional PyPI-ready documentation
+- 🔍 **Example Clarity**: Each example shows input, process, and output
+
+### Technical Details
+- Modified `mem_llm/mem_agent.py` - removed FileHandler from logging setup
+- Rewrote `README.md` with 5 detailed examples
+- Created `examples/07_document_config.py` for PDF/DOCX/TXT feature
+- Verified config changes work correctly with manual YAML edits
+
+## [1.0.8] - 2025-10-20
+
+### Added
+- 🎯 **CLI Tool**: Full-featured command-line interface
+  - `mem-llm chat` - Interactive chat sessions
+  - `mem-llm check` - System verification
+  - `mem-llm stats` - Statistics and analytics
+  - `mem-llm export` - Data export (JSON/TXT)
+  - `mem-llm clear` - User data deletion
+- 📊 **Feature Comparison Matrix**: Clear comparison between JSON and SQL modes
+- 📦 **Improved Dependencies**: Proper separation of core, dev, and optional requirements
+  - `requirements.txt` - Core dependencies only
+  - `requirements-dev.txt` - Development tools
+  - `requirements-optional.txt` - Optional features (web, API, etc.)
+- 🔧 **Better Error Handling**: Improved startup checks with user-friendly messages
+- 📚 **Enhanced Documentation**: CLI usage examples and feature matrices
+
+### Changed
+- 🌍 **Multi-language Support**: Changed from "Turkish Support" to general multi-language
+- 📖 **Documentation**: All content now in English for broader accessibility
+- 🎨 **CLI Entry Point**: Added `mem-llm` console script in setup.py
+
+### Fixed
+- 🐛 Missing `click` dependency in requirements
+- 🐛 Improved error messages when Ollama is not running
+
+### Improved
+- ⚡ Better user experience with CLI commands
+- 📝 Clearer README with usage examples
+- 🎯 More intuitive API design
+
+## [1.0.4] - 2025-10-13
+
+### Added
+- ✨ Config-free knowledge base support - KB now works without config.yaml
+- ✨ Smart keyword extraction for knowledge base search (Turkish & English stopwords)
+- ✨ Enhanced KB context injection - KB data injected directly into user message
+- ✨ Automatic user profile extraction (name, favorite_food, location)
+- ✨ Turkish language support for profile extraction
+- ✨ SQL-JSON memory compatibility methods
+- 📚 New example: `example_knowledge_base.py`
+- 🧪 Comprehensive test suite
+
+### Fixed
+- 🐛 Knowledge base not being used without config.yaml
+- 🐛 LLM ignoring knowledge base information
+- 🐛 User profiles returning empty dictionaries
+- 🐛 Profile updates not working correctly with SQL memory
+- 🐛 Keyword search failing with Turkish queries
+
+### Improved
+- ⚡ Better KB-first response priority in system prompts
+- ⚡ More accurate answers from knowledge base
+- ⚡ Enhanced search algorithm with stopword filtering
+
+## [1.0.3] - 2025-10-12
+
+### Added
+- 📦 Initial PyPI release
+- 🎯 Core memory features (JSON & SQL)
+- 🤖 Ollama integration
+- 💾 Knowledge base system
+- 🛠️ User tools
+- ⚙️ Configuration management
+
+### Features
+- Memory-enabled AI agent
+- JSON and SQL memory backends
+- Knowledge base integration
+- User profile management
+- Conversation history
+- Configuration from YAML/documents
+
+## [1.0.2] - 2025-10-11
+
+### Internal
+- 🔧 Package structure improvements
+- 📝 Documentation updates
+
+## [1.0.1] - 2025-10-10
+
+### Fixed
+- 🐛 Import errors after package rename
+- 📦 Package directory naming issues
+
+## [1.0.0] - 2025-10-09
+
+### Initial Release
+- 🎉 First stable release
+- 🤖 Memory-enabled AI assistant
+- 💾 JSON memory management
+- 🔌 Ollama integration

{mem_llm-1.0.7 → mem_llm-1.0.11}/MANIFEST.in

@@ -1,11 +1,8 @@
 include README.md
 include requirements.txt
+include requirements-dev.txt
+include requirements-optional.txt
 include CHANGELOG.md
-include QUICKSTART.md
-include QUICKSTART_TR.md
-include INTEGRATION_GUIDE.md
-include STRUCTURE.md
 recursive-include mem_llm *.yaml
 recursive-include mem_llm *.example
-recursive-include docs *.md
 

mem_llm-1.0.11/PKG-INFO

@@ -0,0 +1,455 @@
+Metadata-Version: 2.2
+Name: mem-llm
+Version: 1.0.11
+Summary: Memory-enabled AI assistant with local LLM support
+Author-email: "C. Emre Karataş" <karatasqemre@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/emredeveloper/Mem-LLM
+Project-URL: Bug Reports, https://github.com/emredeveloper/Mem-LLM/issues
+Project-URL: Source, https://github.com/emredeveloper/Mem-LLM
+Keywords: llm,ai,memory,agent,chatbot,ollama,local
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+Requires-Dist: requests>=2.31.0
+Requires-Dist: pyyaml>=6.0.1
+Requires-Dist: click>=8.1.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
+Requires-Dist: black>=23.7.0; extra == "dev"
+Requires-Dist: flake8>=6.1.0; extra == "dev"
+Provides-Extra: web
+Requires-Dist: flask>=3.0.0; extra == "web"
+Requires-Dist: flask-cors>=4.0.0; extra == "web"
+Provides-Extra: api
+Requires-Dist: fastapi>=0.104.0; extra == "api"
+Requires-Dist: uvicorn>=0.24.0; extra == "api"
+
+# 🧠 Mem-LLM
+
+[![PyPI version](https://badge.fury.io/py/mem-llm.svg)](https://badge.fury.io/py/mem-llm)
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+**Memory-enabled AI assistant with local LLM support**
+
+Mem-LLM is a powerful Python library that brings persistent memory capabilities to local Large Language Models. Build AI assistants that remember user interactions, manage knowledge bases, and work completely offline with Ollama.
+
+## ✨ Key Features
+
+- 🧠 **Persistent Memory** - Remembers conversations across sessions
+- 🤖 **Universal Ollama Support** - Works with ALL Ollama models (Qwen3, DeepSeek, Llama3, Granite, etc.)
+- 💾 **Dual Storage Modes** - JSON (simple) or SQLite (advanced) memory backends
+- 📚 **Knowledge Base** - Built-in FAQ/support system with categorized entries
+- 🎯 **Dynamic Prompts** - Context-aware system prompts that adapt to active features
+- 👥 **Multi-User Support** - Separate memory spaces for different users
+- 🔧 **Memory Tools** - Search, export, and manage stored memories
+- 🎨 **Flexible Configuration** - Personal or business usage modes
+- 📊 **Production Ready** - Comprehensive test suite with 34+ automated tests
+- 🔒 **100% Local & Private** - No cloud dependencies, your data stays yours
+
+## 🚀 Quick Start
+
+### Installation
+
+```bash
+pip install mem-llm
+```
+
+### Prerequisites
+
+Install and start [Ollama](https://ollama.ai):
+
+```bash
+# Install Ollama (visit https://ollama.ai)
+# Then pull a model
+ollama pull granite4:tiny-h
+
+# Start Ollama service
+ollama serve
+```
+
+### Basic Usage
+
+```python
+from mem_llm import MemAgent
+
+# Create an agent
+agent = MemAgent(model="granite4:tiny-h")
+
+# Set user and chat
+agent.set_user("alice")
+response = agent.chat("My name is Alice and I love Python!")
+print(response)
+
+# Memory persists across sessions
+response = agent.chat("What's my name and what do I love?")
+print(response)  # Agent remembers: "Your name is Alice and you love Python!"
+```
+
+That's it! Just 5 lines of code to get started.
+
+## 📖 Usage Examples
+
+### Multi-User Conversations
+
+```python
+from mem_llm import MemAgent
+
+agent = MemAgent()
+
+# User 1
+agent.set_user("alice")
+agent.chat("I'm a Python developer")
+
+# User 2
+agent.set_user("bob")
+agent.chat("I'm a JavaScript developer")
+
+# Each user has separate memory
+agent.set_user("alice")
+response = agent.chat("What do I do?")  # "You're a Python developer"
+```
+
+### Advanced Configuration
+
+```python
+from mem_llm import MemAgent
+
+# Use SQL database with knowledge base
+agent = MemAgent(
+    model="qwen3:8b",
+    use_sql=True,
+    load_knowledge_base=True,
+    config_file="config.yaml"
+)
+
+# Add knowledge base entry
+agent.add_kb_entry(
+    category="FAQ",
+    question="What are your hours?",
+    answer="We're open 9 AM - 5 PM EST, Monday-Friday"
+)
+
+# Agent will use KB to answer
+response = agent.chat("When are you open?")
+```
+
+### Memory Tools
+
+```python
+from mem_llm import MemAgent
+
+agent = MemAgent(use_sql=True)
+agent.set_user("alice")
+
+# Chat with memory
+agent.chat("I live in New York")
+agent.chat("I work as a data scientist")
+
+# Search memories
+results = agent.search_memories("location")
+print(results)  # Finds "New York" memory
+
+# Export all data
+data = agent.export_user_data()
+print(f"Total memories: {len(data['memories'])}")
+
+# Get statistics
+stats = agent.get_memory_stats()
+print(f"Users: {stats['total_users']}, Memories: {stats['total_memories']}")
+```
+
+### CLI Interface
+
+```bash
+# Interactive chat
+mem-llm chat
+
+# With specific model
+mem-llm chat --model llama3:8b
+
+# Customer service mode
+mem-llm customer-service
+
+# Knowledge base management
+mem-llm kb add --category "FAQ" --question "How to install?" --answer "Run: pip install mem-llm"
+mem-llm kb list
+mem-llm kb search "install"
+```
+
+## 🎯 Usage Modes
+
+### Personal Mode (Default)
+- Single user with JSON storage
+- Simple and lightweight
+- Perfect for personal projects
+- No configuration needed
+
+```python
+agent = MemAgent()  # Automatically uses personal mode
+```
+
+### Business Mode
+- Multi-user with SQL database
+- Knowledge base support
+- Advanced memory tools
+- Requires configuration file
+
+```python
+agent = MemAgent(
+    config_file="config.yaml",
+    use_sql=True,
+    load_knowledge_base=True
+)
+```
+
+## 🔧 Configuration
+
+Create a `config.yaml` file for advanced features:
+
+```yaml
+# Usage mode: 'personal' or 'business'
+usage_mode: business
+
+# LLM settings
+llm:
+  model: granite4:tiny-h
+  base_url: http://localhost:11434
+  temperature: 0.7
+  max_tokens: 2000
+
+# Memory settings
+memory:
+  type: sql  # or 'json'
+  db_path: ./data/memory.db
+  
+# Knowledge base
+knowledge_base:
+  enabled: true
+  kb_path: ./data/knowledge_base.db
+
+# Logging
+logging:
+  level: INFO
+  file: logs/mem_llm.log
+```
+
+## 🧪 Supported Models
+
+Mem-LLM works with **ALL Ollama models**, including:
+
+- ✅ **Thinking Models**: Qwen3, DeepSeek, QwQ
+- ✅ **Standard Models**: Llama3, Granite, Phi, Mistral
+- ✅ **Specialized Models**: CodeLlama, Vicuna, Neural-Chat
+- ✅ **Any Custom Model** in your Ollama library
+
+### Model Compatibility Features
+- 🔄 Automatic thinking mode detection
+- 🎯 Dynamic prompt adaptation
+- ⚡ Token limit optimization (2000 tokens)
+- 🔧 Automatic retry on empty responses
+
+## 📚 Architecture
+
+```
+mem-llm/
+├── mem_llm/
+│   ├── mem_agent.py           # Main agent class
+│   ├── memory_manager.py      # JSON memory backend
+│   ├── memory_db.py           # SQL memory backend
+│   ├── llm_client.py          # Ollama API client
+│   ├── knowledge_loader.py    # Knowledge base system
+│   ├── dynamic_prompt.py      # Context-aware prompts
+│   ├── memory_tools.py        # Memory management tools
+│   ├── config_manager.py      # Configuration handler
+│   └── cli.py                 # Command-line interface
+└── examples/                  # Usage examples
+```
+
+## 🔥 Advanced Features
+
+### Dynamic Prompt System
+Prevents hallucinations by only including instructions for enabled features:
+
+```python
+agent = MemAgent(use_sql=True, load_knowledge_base=True)
+# Agent automatically knows:
+# ✅ Knowledge Base is available
+# ✅ Memory tools are available
+# ✅ SQL storage is active
+```
+
+### Knowledge Base Categories
+Organize knowledge by category:
+
+```python
+agent.add_kb_entry(category="FAQ", question="...", answer="...")
+agent.add_kb_entry(category="Technical", question="...", answer="...")
+agent.add_kb_entry(category="Billing", question="...", answer="...")
+```
+
+### Memory Search & Export
+Powerful memory management:
+
+```python
+# Search across all memories
+results = agent.search_memories("python", limit=5)
+
+# Export everything
+data = agent.export_user_data()
+
+# Get insights
+stats = agent.get_memory_stats()
+```
+
+## 📦 Project Structure
+
+### Core Components
+- **MemAgent**: Main interface for building AI assistants
+- **MemoryManager**: JSON-based memory storage (simple)
+- **SQLMemoryManager**: SQLite-based storage (advanced)
+- **OllamaClient**: LLM communication handler
+- **KnowledgeLoader**: Knowledge base management
+
+### Optional Features
+- **MemoryTools**: Search, export, statistics
+- **ConfigManager**: YAML configuration
+- **CLI**: Command-line interface
+
+## 🧪 Testing
+
+Run the comprehensive test suite:
+
+```bash
+# Install dev dependencies
+pip install -r requirements-dev.txt
+
+# Run all tests (34+ automated tests)
+cd tests
+python run_all_tests.py
+
+# Run specific test
+python -m pytest test_mem_agent.py -v
+```
+
+### Test Coverage
+- ✅ Core imports and dependencies
+- ✅ CLI functionality
+- ✅ Ollama connection and models
+- ✅ JSON memory operations
+- ✅ SQL memory operations
+- ✅ MemAgent features
+- ✅ Configuration management
+- ✅ Multi-user scenarios
+- ✅ Hallucination detection
+
+## 📝 Examples
+
+The `examples/` directory contains ready-to-run demonstrations:
+
+1. **01_hello_world.py** - Simplest possible example (5 lines)
+2. **02_basic_memory.py** - Memory persistence basics
+3. **03_multi_user.py** - Multiple users with separate memories
+4. **04_customer_service.py** - Real-world customer service scenario
+5. **05_knowledge_base.py** - FAQ/support system
+6. **06_cli_demo.py** - Command-line interface examples
+7. **07_document_config.py** - Configuration from documents
+
+## 🛠️ Development
+
+### Setup Development Environment
+
+```bash
+git clone https://github.com/emredeveloper/Mem-LLM.git
+cd Mem-LLM
+pip install -e .
+pip install -r requirements-dev.txt
+```
+
+### Running Tests
+
+```bash
+pytest tests/ -v --cov=mem_llm
+```
+
+### Building Package
+
+```bash
+python -m build
+twine upload dist/*
+```
+
+## 📋 Requirements
+
+### Core Dependencies
+- Python 3.8+
+- requests>=2.31.0
+- pyyaml>=6.0.1
+- click>=8.1.0
+
+### Optional Dependencies
+- pytest>=7.4.0 (for testing)
+- flask>=3.0.0 (for web interface)
+- fastapi>=0.104.0 (for API server)
+
+## 🤝 Contributing
+
+Contributions are welcome! Please feel free to submit a Pull Request.
+
+1. Fork the repository
+2. Create your feature branch (`git checkout -b feature/AmazingFeature`)
+3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
+4. Push to the branch (`git push origin feature/AmazingFeature`)
+5. Open a Pull Request
+
+## 📄 License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
+
+## 👤 Author
+
+**C. Emre Karataş**
+- Email: karatasqemre@gmail.com
+- GitHub: [@emredeveloper](https://github.com/emredeveloper)
+
+## 🙏 Acknowledgments
+
+- Built with [Ollama](https://ollama.ai) for local LLM support
+- Inspired by the need for privacy-focused AI assistants
+- Thanks to all contributors and users
+
+## 📊 Project Status
+
+- **Version**: 1.0.10
+- **Status**: Beta (Production Ready)
+- **Last Updated**: October 20, 2025
+
+## 🔗 Links
+
+- **PyPI**: https://pypi.org/project/mem-llm/
+- **GitHub**: https://github.com/emredeveloper/Mem-LLM
+- **Issues**: https://github.com/emredeveloper/Mem-LLM/issues
+- **Documentation**: See examples/ directory
+
+## 📈 Roadmap
+
+- [ ] Web UI dashboard
+- [ ] REST API server
+- [ ] Vector database integration
+- [ ] Multi-language support
+- [ ] Cloud backup options
+- [ ] Advanced analytics
+
+---
+
+**⭐ If you find this project useful, please give it a star on GitHub!**