vector-task-mcp 1.3.2__tar.gz → 1.4.0__tar.gz

{vector_task_mcp-1.3.2 → vector_task_mcp-1.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: vector-task-mcp
-Version: 1.3.2
+Version: 1.4.0
 Summary: A secure, vector-based task management server for Claude Desktop using sqlite-vec and sentence-transformers
 Author-email: Xsaven <xsaven@gmail.com>
 License-Expression: MIT
@@ -34,11 +34,15 @@ A **secure, vector-based task management server** for Claude Desktop using `sqli
 - **💾 Persistent Storage**: SQLite database with vector indexing via `sqlite-vec`
 - **🏷️ Smart Organization**: Priorities, tags, and subtasks for better task management
 - **📋 Task Lifecycle**: Track tasks from pending → in_progress → completed → tested → validated (or stopped)
+- **🔐 Tag Normalization**: Automatic tag deduplication with semantic similarity
+- **📊 IDF Weights**: Rare tags boost search relevance more than common tags
+- **🎯 Tag Classification**: Filter tags vs boost tags for smart ranking
+- **🔄 Alias Scent**: Original tag variants preserved for search context
 - **🔒 Security First**: Input validation, path sanitization, and resource limits
 - **⚡ High Performance**: Fast embedding generation with `sentence-transformers`
-- **📊 Rich Statistics**: Comprehensive task analytics and progress tracking
+- **📈 Rich Statistics**: Comprehensive task analytics and progress tracking
 - **🔄 Hierarchical Tasks**: Support for parent-child task relationships
-- **📈 Priority Management**: Organize tasks by priority (low, medium, high, critical)
+- **📊 Priority Management**: Organize tasks by priority (low, medium, high, critical)
 - **💬 Task Comments**: Add notes and updates to tasks without changing content
 
 ## 🛠️ Technical Stack
@@ -48,6 +52,7 @@ A **secure, vector-based task management server** for Claude Desktop using `sqli
 | **Vector DB** | sqlite-vec | Vector storage and similarity search |
 | **Embeddings** | sentence-transformers/all-MiniLM-L6-v2 | 384D text embeddings |
 | **MCP Framework** | FastMCP | High-level tools-only server |
+| **Tag Normalization** | Custom (src/normalization.py) | Semantic tag deduplication |
 | **Dependencies** | uv script headers | Self-contained deployment |
 | **Security** | Custom validation | Path/input sanitization |
 | **Testing** | pytest + coverage | Comprehensive test suite |
@@ -67,7 +72,13 @@ vector-task-mcp/
 │   ├── __init__.py                    # Package initialization
 │   ├── models.py                      # Data models & configuration
 │   ├── security.py                    # Security validation & sanitization
-│   └── task_store.py                  # SQLite-vec task operations
+│   ├── task_store.py                  # SQLite-vec task operations
+│   ├── embeddings.py                  # Embedding model wrapper
+│   └── normalization.py               # Tag normalization & classification
+│
+├── tests/                             # Test suite
+│   ├── test_task_store.py            # Task store tests
+│   └── test_normalization.py         # Normalization tests
 │
 └── .gitignore                         # Git exclusions
 ```
@@ -345,6 +356,89 @@ Returns:
 }
 ```
 
+### Tag Normalization Tools
+
+**19. `tag_normalize_preview` - Preview Tag Merges**
+```
+Preview which tags can be merged:
+- threshold: 0.90 (strict) or 0.85 (aggressive)
+```
+
+Shows similar tags that can be merged into canonical forms.
+
+**20. `tag_normalize_apply` - Apply Tag Normalization**
+```
+Apply tag normalization with optional dry_run
+```
+
+Merges variant tags into canonical forms and stores original variants in `tag_variants`.
+
+**21. `tag_similarity` - Compare Two Tags**
+```
+Compare similarity between "auth" and "authentication"
+```
+
+Returns cosine similarity score (0.0-1.0).
+
+**22. `canonical_tag_add` - Add Canonical Mapping**
+```
+Add mapping: "authentication" → "auth"
+```
+
+**23. `canonical_tag_remove` - Remove Mapping**
+```
+Remove mapping for "authentication"
+```
+
+**24. `canonical_tag_list` - List All Mappings**
+```
+List all canonical tag mappings
+```
+
+**25. `get_canonical_tags` - List Canonical Tags**
+```
+List all canonical tags only
+```
+
+### Tag Intelligence Tools
+
+**26. `tag_frequencies` - Get Tag Frequencies & IDF Weights**
+```
+Get tag frequencies with IDF weights
+```
+
+Returns frequency statistics and IDF weights for search ranking:
+```json
+{
+  "api": {"count": 10, "frequency": 0.4, "idf_weight": 0.621},
+  "vendor:stripe": {"count": 1, "frequency": 0.04, "idf_weight": 1.443}
+}
+```
+
+**27. `tag_weights` - Get Simplified IDF Weights**
+```
+Get IDF weights for all tags (for search ranking)
+```
+
+**28. `tag_classify` - Classify Single Tag**
+```
+Classify tag "vendor:stripe"
+```
+
+Returns boost level (high/medium/low/filter_only) for ranking.
+
+**29. `tags_classify_batch` - Classify Multiple Tags**
+```
+Classify tags: ["vendor:stripe", "api", "status:pending"]
+```
+
+**30. `search_explain` - Search with Ranking Explanation**
+```
+Search for "authentication" with ranking explanation
+```
+
+Shows how IDF weights, classification, and variants affect ranking.
+
 ### Task Priorities
 
 | Priority | Use Cases |
@@ -392,11 +486,23 @@ uv run main.py --working-dir ~/projects/my-project
 
 ```
 your-project/
-├── tasks.db                # SQLite database with task vectors
-├── src/                    # Your project files
+├── memory/
+│   └── tasks.db              # SQLite database with task vectors
+├── src/                      # Your project files
 └── other-files...
 ```
 
+### Database Schema
+
+**tasks table**:
+- Core task data + `tags` (canonical) + `tag_variants` (original variants)
+
+**canonical_tags table**:
+- Predefined tag mappings (variant → canonical)
+
+**task_vectors table**:
+- 384-dimensional embeddings for semantic search
+
 ### Security Limits
 
 - **Max task content**: 10,000 characters
@@ -449,6 +555,100 @@ your-project/
 "Refactor legacy authentication module to use new security library"
 ```
 
+## 🏷️ Tag Normalization
+
+### Overview
+
+Tag normalization reduces tag fragmentation by merging semantically similar tags:
+
+| Before | After |
+|--------|-------|
+| auth, authentication, auth-api, login | → **auth** |
+| db, database, database-setup | → **database** |
+| api, rest api, API | → **api** |
+
+### Hard Guards (Prevent Wrong Merges)
+
+| Guard | Rule | Example |
+|-------|------|---------|
+| **Version** | Different versions → NO | `php8` ≠ `php7` |
+| **Numeric** | Different numbers → NO | `api1` ≠ `api2` |
+| **Facet** | Different prefixes → NO | `type:*` ≠ `domain:*` |
+| **Prefix** | Structured ≠ Plain | `type:refactor` ≠ `refactor` |
+
+### Substring Boost
+
+Tags that are substrings get a small boost if:
+- Shorter word ≥ 4 characters
+- Not in stop-words (api, ui, db, etc.)
+
+Example: `"laravel"` ⊂ `"laravel framework"` → boost to 0.95
+
+### Facet Model (Colon Tags)
+
+Tags with colons (`prefix:value`) are treated as structured facets:
+
+```
+type:refactor     ← facet: "type", value: "refactor"
+vendor:stripe     ← facet: "vendor", value: "stripe"
+module:terminal   ← facet: "module", value: "terminal"
+```
+
+Rules:
+- Same prefix can merge if similar: `type:refactor` ↔ `type:refactoring` ✅
+- Different prefixes never merge: `type:*` ↔ `domain:*` ❌
+- Structured never merges with plain: `type:*` ↔ `refactor` ❌
+
+### Tag Variants (Alias Scent)
+
+When tags are migrated, original variants are preserved:
+
+```json
+{
+  "tags": ["auth"],
+  "tag_variants": ["authentication", "auth-api", "login"]
+}
+```
+
+Variants provide:
+- Context for search ranking
+- Explanation in UI ("Why auth? Because was login/authentication")
+- Rerank signal for queries
+
+## 📊 IDF Weights & Tag Classification
+
+### IDF (Inverse Document Frequency)
+
+Rare tags boost relevance more than common tags:
+
+```
+idf_weight = 1 / log(1 + frequency)
+```
+
+| Tag | Count | IDF Weight | Effect |
+|-----|-------|------------|--------|
+| `api` | 70% of tasks | 0.38 | Low signal |
+| `vendor:stripe` | 3% of tasks | 1.44 | Strong signal |
+
+### Tag Classification
+
+Tags are classified by boost level:
+
+| Level | Boost | Examples |
+|-------|-------|----------|
+| `high` | 1.5 | `vendor:*`, `module:*`, `service:*` |
+| `medium` | 1.0 | Facet tags (`domain:*`, `type:*`), specific tags |
+| `low` | 0.5 | General tags (`api`, `backend`, `test`) |
+| `filter_only` | 0.1 | `status:*`, `priority:*` |
+
+### Search Ranking
+
+Final search score combines:
+1. **Vector similarity** (cosine distance)
+2. **IDF weight** (rare tags boost more)
+3. **Tag classification** (high > medium > low > filter)
+4. **Variant bonus** (tasks with tag_variants get small boost)
+
 ## 🔍 How Semantic Search Works
 
 The server uses **sentence-transformers** to convert tasks into 384-dimensional vectors that capture semantic meaning:
@@ -629,6 +829,33 @@ Based on testing with various dataset sizes:
 
 *Tested on MacBook Air M1 with sentence-transformers/all-MiniLM-L6-v2*
 
+## 🧪 Test Coverage
+
+```
+tests/
+├── test_task_store.py     # 60 tests - Task store operations
+└── test_normalization.py  # 45 tests - Tag normalization
+
+Total: 105 tests
+```
+
+Run tests:
+```bash
+uv run pytest tests/ -v
+```
+
+## 🔄 Backward Compatibility
+
+New features are backward compatible:
+
+| Feature | Migration |
+|---------|-----------|
+| `tag_variants` column | Auto-added via ALTER TABLE |
+| `canonical_tags` table | Auto-created via CREATE TABLE IF NOT EXISTS |
+| IDF reranking | Opt-in via `use_idf_rerank=True` |
+
+Existing databases work without changes. New columns/tables added automatically on first run.
+
 ## 🤝 Contributing
 
 This is a standalone MCP server designed for personal/team use. For improvements: