memoryforge 1.0.0__tar.gz

memoryforge-1.0.0/.gitignore

@@ -0,0 +1 @@
+* .pdf

memoryforge-1.0.0/ARCHITECTURE.md

@@ -0,0 +1,332 @@
+# MemoryForge Architecture
+
+## System Overview
+
+```
+┌────────────────────────────────────────────────────────────────────────────┐
+│                            MemoryForge v1.0                                │
+├────────────────────────────────────────────────────────────────────────────┤
+│                                                                            │
+│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐                 │
+│  │     CLI      │    │  MCP Server  │    │  Python API  │                 │
+│  │   (click)    │    │   (stdio)    │    │   (direct)   │                 │
+│  └──────┬───────┘    └──────┬───────┘    └──────┬───────┘                 │
+│         │                   │                   │                          │
+│         └───────────────────┼───────────────────┘                          │
+│                             ▼                                              │
+│  ┌─────────────────────────────────────────────────────────────────────┐  │
+│  │                      Core Services Layer                            │  │
+│  │  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌──────────────┐  │  │
+│  │  │   Memory    │ │  Retrieval  │ │   Project   │ │     Git      │  │  │
+│  │  │   Manager   │ │   Engine    │ │   Router    │ │ Integration  │  │  │
+│  │  └─────────────┘ └─────────────┘ └─────────────┘ └──────────────┘  │  │
+│  │  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐                   │  │
+│  │  │ Validation  │ │Consolidation│ │   Sync      │                   │  │
+│  │  │   Layer     │ │   Engine    │ │  Manager    │                   │  │
+│  │  └─────────────┘ └─────────────┘ └─────────────┘                   │  │
+│  └─────────────────────────────────────────────────────────────────────┘  │
+│                             │                                              │
+│         ┌───────────────────┼───────────────────┐                          │
+│         ▼                   ▼                   ▼                          │
+│  ┌─────────────┐     ┌─────────────┐     ┌─────────────┐                  │
+│  │   SQLite    │     │   Qdrant    │     │  Embedding  │                  │
+│  │  (Primary)  │◄────│  (Vectors)  │◄────│   Factory   │                  │
+│  └─────────────┘     └─────────────┘     └──────┬──────┘                  │
+│                                                  │                          │
+│                              ┌───────────────────┼───────────────────┐     │
+│                              ▼                                       ▼     │
+│                       ┌─────────────┐                         ┌──────────┐ │
+│                       │    Local    │                         │  OpenAI  │ │
+│                       │ (MiniLM-L6) │                         │   API    │ │
+│                       └─────────────┘                         └──────────┘ │
+│                                                                            │
+└────────────────────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## Directory Structure
+
+```
+memoryforge/
+├── __init__.py              # Package exports
+├── __main__.py              # Entry point
+├── cli.py                   # CLI commands (click)
+├── config.py                # Configuration management
+├── models.py                # Pydantic data models
+├── migrate.py               # Database migration logic
+│
+├── core/                    # Core business logic
+│   ├── memory_manager.py    # CRUD operations for memories
+│   ├── retrieval.py         # Semantic search engine
+│   ├── validation.py        # Input validation rules
+│   ├── project_router.py    # Multi-project management
+│   ├── memory_consolidator.py # Memory merging
+│   ├── embedding_factory.py # Embedding provider selector
+│   ├── embedding_service.py # OpenAI embeddings
+│   ├── local_embedding_service.py  # sentence-transformers
+│   ├── git_integration.py   # Git-memory links
+│   └── git_scanner.py       # Architectural commit detection
+│
+├── storage/                 # Persistence layer
+│   ├── sqlite_db.py         # SQLite operations
+│   └── qdrant_store.py      # Vector store operations
+│
+├── sync/                    # Team sync (v2.1)
+│   ├── adapter.py           # Sync backend interface
+│   ├── encryption.py        # AES-256 encryption
+│   ├── local_file_adapter.py # File-based sync
+│   └── manager.py           # Sync orchestration
+│
+└── mcp/                     # MCP integration
+    ├── __init__.py
+    └── server.py            # MCP server implementation
+```
+
+---
+
+## Data Flow
+
+### Memory Creation
+
+```
+User Input
+    │
+    ▼
+┌─────────────┐
+│  Validation │  ← Type, content length, format checks
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   SQLite    │  ← Memory stored (confirmed=false)
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   Confirm   │  ← User confirms memory is correct
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│  Embedding  │  ← Generate vector (local or OpenAI)
+│   Factory   │
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   Qdrant    │  ← Store vector with memory ID
+└─────────────┘
+```
+
+### Semantic Search
+
+```
+Search Query
+    │
+    ▼
+┌─────────────┐
+│  Embedding  │  ← Query → Vector
+│   Factory   │
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   Qdrant    │  ← ANN search, return top-k IDs
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   SQLite    │  ← Fetch full memory data
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│  Retrieval  │  ← Score, rank, filter stale
+│   Engine    │
+└──────┬──────┘
+       │
+       ▼
+Search Results
+```
+
+---
+
+## Core Components
+
+### Memory Manager
+**File:** `core/memory_manager.py`
+
+Handles CRUD operations for memories:
+- Create (unconfirmed) → Confirm → Generate embedding
+- Update with version history
+- Delete (removes from SQLite and Qdrant)
+
+### Retrieval Engine
+**File:** `core/retrieval.py`
+
+Semantic search with:
+- Query embedding generation
+- Qdrant similarity search
+- Staleness filtering
+- Score-based ranking
+
+### Project Router
+**File:** `core/project_router.py`
+
+Multi-project support:
+- Project creation/switching
+- Path-based auto-detection
+- Separate Qdrant collections per project
+
+### Memory Consolidator
+**File:** `core/memory_consolidator.py`
+
+Memory merging:
+- Similarity detection (threshold: 0.90)
+- Archive originals with rollback support
+- Stats tracking
+
+### Embedding Factory
+**File:** `core/embedding_factory.py`
+
+Provider abstraction:
+- Local: `sentence-transformers` (all-MiniLM-L6-v2, 384 dims)
+- OpenAI: text-embedding-3-small (1536 dims)
+
+---
+
+## Storage Layer
+
+### SQLite (Primary Store)
+**File:** `storage/sqlite_db.py`
+
+Source of truth for all data:
+
+| Table | Description |
+|-------|-------------|
+| `projects` | Project metadata |
+| `memories` | Memory content, type, flags |
+| `memory_versions` | Content history for rollback |
+| `memory_links` | Git commit associations |
+| `embeddings` | Memory-to-vector-ID mapping |
+
+### Qdrant (Derived Store)
+**File:** `storage/qdrant_store.py`
+
+Vector storage (can be rebuilt from SQLite):
+- One collection per project
+- Dimension matches embedding provider
+- Supports filtered search
+
+---
+
+## Data Models
+
+**File:** `models.py`
+
+```python
+class MemoryType(Enum):
+    STACK = "stack"           # Languages, frameworks, libraries
+    DECISION = "decision"     # Architecture decisions
+    CONSTRAINT = "constraint" # Performance, deadlines
+    CONVENTION = "convention" # Code conventions
+    NOTE = "note"             # General notes
+
+class Memory(BaseModel):
+    id: UUID
+    content: str              # Max 10KB
+    type: MemoryType
+    project_id: UUID
+    confirmed: bool
+    is_stale: bool
+    is_archived: bool
+    created_at: datetime
+    last_accessed: datetime   # Updated on retrieval
+```
+
+---
+
+## MCP Integration
+
+**File:** `mcp/server.py`
+
+Exposes tools via Model Context Protocol:
+
+| Tool | Operation |
+|------|-----------|
+| `store_memory` | Create + optional confirm |
+| `search_memory` | Semantic search |
+| `list_memory` | Recent memories |
+| `delete_memory` | Remove by ID |
+| `memory_timeline` | Chronological view |
+| `list_projects` | All projects |
+| `switch_project` | Change active project |
+| `project_status` | Current project info |
+
+---
+
+## Team Sync
+
+**Files:** `sync/manager.py`, `sync/encryption.py`
+
+Encrypted sync to shared storage:
+
+```
+Local SQLite
+    │
+    ▼
+┌─────────────┐
+│  Export     │  ← Serialize memories to JSON
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│ Encryption  │  ← AES-256-GCM
+└──────┬──────┘
+       │
+       ▼
+┌─────────────┐
+│   Adapter   │  ← Write to shared path
+└─────────────┘
+```
+
+Conflict detection via timestamps and checksums.
+
+---
+
+## Performance
+
+| Operation | Latency | Notes |
+|-----------|---------|-------|
+| Memory creation | <50ms | SQLite write |
+| Confirmation | 100-500ms | Embedding + Qdrant |
+| Semantic search | 50-200ms | Depends on collection size |
+| Max memories | ~100K | Per project, limited by RAM |
+
+### Embedding Providers
+
+| Provider | Dimensions | Speed | Cost |
+|----------|------------|-------|------|
+| Local (MiniLM) | 384 | Fast | Free |
+| OpenAI (3-small) | 1536 | Network | ~$0.02/1K |
+
+---
+
+## Key Design Decisions
+
+1. **SQLite as Source of Truth**
+   - All data restorable from SQLite
+   - Qdrant is derived (can rebuild via `reindex`)
+
+2. **Two-Phase Confirmation**
+   - Memories created as unconfirmed
+   - Embedding only on confirmation (saves cost/compute)
+
+3. **Local-First Architecture**
+   - All data stored locally by default
+   - Sync is opt-in, encrypted
+
+4. **Provider Abstraction**
+   - Switch embedding providers without code changes
+   - Dimension mismatch requires reindex

memoryforge-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 MemoryForge Team
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

memoryforge-1.0.0/PKG-INFO

@@ -0,0 +1,314 @@
+Metadata-Version: 2.4
+Name: memoryforge
+Version: 1.0.0
+Summary: Local-first memory layer for AI coding assistants
+Project-URL: Homepage, https://memoryforge.dev
+Project-URL: Repository, https://github.com/memoryforge/core
+Project-URL: Documentation, https://docs.memoryforge.dev
+Author: MemoryForge Team
+License-Expression: MIT
+License-File: LICENSE
+Keywords: ai,coding-assistant,mcp,memory,vector-database
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.11
+Requires-Dist: click>=8.1.0
+Requires-Dist: mcp>=1.0.0
+Requires-Dist: pydantic-settings>=2.0.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: pyyaml>=6.0.0
+Requires-Dist: qdrant-client>=1.7.0
+Requires-Dist: rich>=13.0.0
+Provides-Extra: all
+Requires-Dist: openai>=1.0.0; extra == 'all'
+Requires-Dist: sentence-transformers>=2.2.0; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: mypy>=1.0.0; extra == 'dev'
+Requires-Dist: openai>=1.0.0; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.21.0; extra == 'dev'
+Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
+Requires-Dist: pytest>=7.0.0; extra == 'dev'
+Requires-Dist: ruff>=0.1.0; extra == 'dev'
+Provides-Extra: local
+Requires-Dist: sentence-transformers>=2.2.0; extra == 'local'
+Provides-Extra: openai
+Requires-Dist: openai>=1.0.0; extra == 'openai'
+Provides-Extra: sync
+Requires-Dist: cryptography>=41.0.0; extra == 'sync'
+Requires-Dist: gitpython>=3.1.0; extra == 'sync'
+Description-Content-Type: text/markdown
+
+# MemoryForge
+
+![Python](https://img.shields.io/badge/python-3.11%2B-blue)
+![Tests](https://img.shields.io/badge/tests-111%20passed-brightgreen)
+![License](https://img.shields.io/badge/license-MIT-green)
+![Version](https://img.shields.io/badge/version-1.0.0-purple)
+
+> **Local-first memory layer for AI coding assistants**
+
+MemoryForge provides persistent, semantic memory for AI pair programmers. Store decisions, constraints, and context that survives across sessions.
+
+---
+
+## Features
+
+| Feature | Description |
+|---------|-------------|
+| **Semantic Search** | Vector-based retrieval using Qdrant |
+| **Memory Types** | Stack, Decision, Constraint, Convention, Note |
+| **Multi-Project** | Separate memory indexes per project |
+| **Consolidation** | Merge similar memories with rollback support |
+| **Staleness Tracking** | Mark outdated memories, filter from search |
+| **Git Integration** | Link memories to commits |
+| **Team Sync** | Encrypted sync to shared storage |
+| **Graph Memory** | Memory-to-memory relationships (v3) |
+| **Confidence Scoring** | Trust scores for prioritized retrieval (v3) |
+| **Conflict Resolution** | Sync conflict detection and logging (v3) |
+| **MCP Server** | IDE integration via Model Context Protocol |
+
+---
+
+## Installation
+
+```bash
+# From GitHub (recommended)
+pip install git+https://github.com/agusain2001/memoryforge.git
+
+# With local embeddings (free, recommended)
+pip install "memoryforge[local] @ git+https://github.com/agusain2001/memoryforge.git"
+
+# With OpenAI embeddings
+pip install "memoryforge[openai] @ git+https://github.com/agusain2001/memoryforge.git"
+
+# With team sync
+pip install "memoryforge[sync] @ git+https://github.com/agusain2001/memoryforge.git"
+
+# Everything
+pip install "memoryforge[all] @ git+https://github.com/agusain2001/memoryforge.git"
+```
+
+**Requirements:** Python 3.11+
+
+---
+
+## Quick Start
+
+```bash
+# Initialize MemoryForge
+cd your-project
+memoryforge init --name "my-project"
+
+# Add a memory
+memoryforge add "Using FastAPI for backend, PostgreSQL for data" --type stack
+
+# Search memories
+memoryforge search "what database?"
+
+# List recent memories
+memoryforge list --limit 5
+```
+
+---
+
+## CLI Reference
+
+### Core Commands
+
+```bash
+memoryforge init [OPTIONS]              # Initialize MemoryForge
+  -n, --name TEXT                       # Project name
+  -p, --provider [local|openai]         # Embedding provider (default: local)
+  -k, --api-key TEXT                    # OpenAI API key (if using openai)
+
+memoryforge add CONTENT [OPTIONS]       # Add a memory
+  -t, --type [stack|decision|constraint|convention|note]
+  --confirm / --no-confirm              # Immediately confirm
+
+memoryforge list [OPTIONS]              # List memories
+  -t, --type TYPE                       # Filter by type
+  -a, --all                             # Include unconfirmed
+  -l, --limit INTEGER                   # Max results
+
+memoryforge search QUERY [OPTIONS]      # Semantic search
+  -t, --type TYPE                       # Filter by type
+  -l, --limit INTEGER                   # Max results
+
+memoryforge confirm MEMORY_ID           # Confirm a pending memory
+memoryforge delete MEMORY_ID            # Delete a memory
+memoryforge timeline [--limit N]        # Chronological view
+memoryforge status                      # Show project status
+memoryforge reindex [--force]           # Rebuild vector index
+memoryforge serve                       # Start MCP server
+```
+
+### Project Management
+
+```bash
+memoryforge project create --name NAME [--path PATH]
+memoryforge project list
+memoryforge project switch NAME_OR_ID
+memoryforge project delete NAME_OR_ID
+```
+
+### Memory Consolidation
+
+```bash
+memoryforge consolidate suggest [--limit N]
+memoryforge consolidate apply ID1 ID2 ... --content "merged content"
+memoryforge consolidate rollback CONSOLIDATED_ID
+memoryforge consolidate stats
+```
+
+### Staleness Management
+
+```bash
+memoryforge stale list
+memoryforge stale mark MEMORY_ID --reason "outdated"
+memoryforge stale clear MEMORY_ID
+memoryforge stale unused [--days N]
+```
+
+### Git Integration
+
+```bash
+memoryforge git status
+memoryforge git sync [--limit N]        # Find architectural commits
+memoryforge git link MEMORY_ID COMMIT_SHA
+memoryforge git activity [--days N]
+```
+
+### Team Sync
+
+```bash
+memoryforge sync init --path ./shared-folder
+memoryforge sync push [--force]
+memoryforge sync pull
+memoryforge sync status
+```
+
+### Database Migration
+
+```bash
+memoryforge migrate [--rollback] [--backup-file FILE]
+```
+
+### Graph Memory (v3)
+
+```bash
+memoryforge graph view MEMORY_ID           # View memory relationships
+memoryforge graph link SRC_ID TGT_ID       # Create relationship
+  -t, --type [caused_by|supersedes|relates_to|blocks|depends_on]
+```
+
+### Conflict Management (v3)
+
+```bash
+memoryforge conflicts list [--memory-id ID]  # List sync conflicts
+memoryforge conflicts show MEMORY_ID         # Detailed conflict history
+```
+
+### Confidence Scoring (v3)
+
+```bash
+memoryforge confidence show MEMORY_ID      # View confidence breakdown
+memoryforge confidence update MEMORY_ID    # Recalculate score
+memoryforge confidence low [--threshold N] # List low-confidence memories
+memoryforge confidence refresh             # Batch update all scores
+```
+
+### Memory Sharing (v3)
+
+```bash
+memoryforge share memory MEMORY_ID          # Share a memory with team
+  --with TEAM                               # Share target (default: team)
+  -n, --note TEXT                           # Add a note for recipients
+memoryforge share list                      # List shared memories
+memoryforge share import FILENAME           # Import shared memory
+```
+
+---
+
+## Configuration
+
+Configuration stored in `~/.memoryforge/config.yaml`:
+
+```yaml
+embedding_provider: local              # "local" or "openai"
+local_embedding_model: all-MiniLM-L6-v2
+openai_api_key: sk-...                 # Required if using openai
+openai_embedding_model: text-embedding-3-small
+max_results: 5
+min_score: 0.5
+active_project_id: <uuid>
+```
+
+> **Note:** Changing `embedding_provider` requires re-indexing: `memoryforge reindex --force`
+
+---
+
+## MCP Server
+
+MemoryForge integrates with AI IDEs via the Model Context Protocol:
+
+```bash
+memoryforge serve
+```
+
+### IDE Configuration (Cursor/VSCode)
+
+```json
+{
+  "mcpServers": {
+    "memoryforge": {
+      "command": "memoryforge",
+      "args": ["serve"]
+    }
+  }
+}
+```
+
+### Available Tools
+
+| Tool | Description |
+|------|-------------|
+| `store_memory` | Store a new memory |
+| `search_memory` | Semantic search |
+| `list_memory` | List recent memories |
+| `delete_memory` | Delete a memory |
+| `memory_timeline` | Chronological view |
+| `list_projects` | List all projects |
+| `switch_project` | Switch active project |
+| `project_status` | Current project info |
+
+---
+
+## Troubleshooting
+
+| Error | Cause | Fix |
+|-------|-------|-----|
+| `vector dimension mismatch` | Embedding provider changed | `rm -rf ~/.memoryforge/qdrant && memoryforge reindex` |
+| `Memory not found in search` | Qdrant indexing failed | `memoryforge confirm <id> --force` |
+| `database is locked` | Multiple processes | Ensure single CLI/MCP instance |
+| `SyncConflictError` | Concurrent team edits | `memoryforge sync push --force` |
+
+---
+
+## Development
+
+```bash
+git clone https://github.com/agusain2001/memoryforge.git
+cd memoryforge
+pip install -e ".[dev]"
+pytest tests/ -v
+```
+
+---
+
+## License
+
+MIT License - see [LICENSE](LICENSE)