sqlew 3.2.5 → 3.6.0

package/docs/MIGRATION_v3.6.0.md

@@ -0,0 +1,170 @@
+# Migration Guide: v3.3.x → v3.6.0
+
+**Release Date**: 2025-10-25
+**Migration Type**: ✅ Automatic (Zero Downtime)
+
+---
+
+## Overview
+
+Version 3.6.0 introduces major improvements:
+- **Knex.js Database Layer**: Cross-database support (SQLite/PostgreSQL/MySQL)
+- **Help System Optimization**: Database-driven queries (60-70% token reduction)
+- **Enhanced Performance**: Optimized query patterns and indexing
+
+Migration is **fully automatic** and **backward compatible**.
+
+### What's New
+
+- Cross-database support with automatic migrations
+- 6 new help query actions (`help_action`, `help_params`, `help_tool`, `help_use_case`, `help_list_use_cases`, `help_next_actions`)
+- 7 new database tables for help system
+- 41 queryable use-cases across 6 categories
+- v3.5.x branch deprecated (migrate directly from v3.3.x)
+
+---
+
+## Migration Checklist
+
+### ✅ Pre-Migration
+
+- [ ] Backup database: `cp .claude/docs/sqlew.db .claude/docs/sqlew.db.backup`
+- [ ] Ensure Node.js 18+
+- [ ] Review [CHANGELOG.md](../CHANGELOG.md)
+
+### ✅ Migration Steps
+
+1. **Update & Rebuild**
+   ```bash
+   git pull origin main
+   npm install
+   npx tsc
+   ```
+
+2. **Start Server**
+   ```bash
+   node dist/index.js --db-path=.claude/docs/sqlew.db
+   ```
+
+   Migration runs automatically:
+   - Detects v3.3.x database
+   - Migrates to Knex.js schema
+   - Creates 7 help system tables
+   - Seeds 41 use-cases + metadata
+   - Completes in <5 seconds
+
+3. **Verify**
+   ```bash
+   # Check migration succeeded
+   sqlite3 .claude/docs/sqlew.db "SELECT COUNT(*) FROM knex_migrations;"
+   # Returns: 4
+   ```
+
+### ✅ Post-Migration
+
+- [ ] Verify existing tools work
+- [ ] Test new help actions (optional)
+
+---
+
+## Breaking Changes
+
+**None** - 100% backward compatible. All existing actions work identically.
+
+---
+
+## New Features
+
+### 6 Help Query Actions (stats tool)
+
+| Action | Purpose | Token Savings |
+|--------|---------|---------------|
+| `help_action` | Action parameters + examples | ~200 tokens (vs ~2,000) |
+| `help_params` | Parameter list only | ~229 tokens (vs ~1,500) |
+| `help_tool` | Tool overview + actions | ~139 tokens (vs ~5,000) |
+| `help_use_case` | Single use-case workflow | ~150 tokens each |
+| `help_list_use_cases` | Filter/list use-cases | ~282-584 tokens |
+| `help_next_actions` | Common next actions | ~63 tokens |
+
+Example:
+```json
+{"tool": "stats", "action": "help_action", "tool": "decision", "action": "set"}
+```
+
+---
+
+## Database Changes
+
+### New Tables (7)
+
+**Master Tables:**
+1. `m_help_tools` - Tool metadata (7 tools)
+2. `m_help_actions` - Actions (41 total)
+3. `m_help_use_case_categories` - Categories (6)
+
+**Transaction Tables:**
+4. `t_help_action_params` - Parameters (98)
+5. `t_help_action_examples` - Examples (41)
+6. `t_help_use_cases` - Use-cases (41)
+7. `t_help_action_sequences` - Common patterns
+
+### Impact
+
+- Size: +210 KB (Knex overhead ~10 KB, help system ~200 KB)
+- Performance: <200ms per help query
+- Knex tracking: `knex_migrations` and `knex_migrations_lock` tables added
+
+---
+
+## Rollback Instructions
+
+**Important**: v3.6.0 uses Knex.js schema - **must restore backup** to rollback.
+
+1. Stop server
+2. Restore backup: `cp .claude/docs/sqlew.db.backup .claude/docs/sqlew.db`
+3. Checkout v3.3.x:
+   ```bash
+   git checkout v3.3.x
+   npm install && npx tsc
+   node dist/index.js
+   ```
+
+**Note**: v3.5.x deprecated - migrate directly v3.3.x → v3.6.0.
+
+---
+
+## FAQ
+
+**Q: Do I need to change my code/workflows?**
+A: No. All existing actions work identically.
+
+**Q: Is the migration reversible?**
+A: Yes, but requires restoring database backup (Knex schema incompatible with v3.3.x).
+
+**Q: What if migration fails?**
+A: Migration is transactional - auto-rollback on failure.
+
+**Q: Can I disable the help system?**
+A: Yes - simply don't use the new `help_*` actions.
+
+**Q: How do I verify migration succeeded?**
+A: Check logs for "✓ Migration complete: 3.3.x → 3.6.0" or run:
+```bash
+sqlite3 .claude/docs/sqlew.db "SELECT COUNT(*) FROM knex_migrations;"
+# Returns: 4
+```
+
+**Q: Performance impact?**
+A: Negligible. Help queries are optional and independent.
+
+---
+
+## Summary
+
+- **Path**: v3.3.x → v3.6.0 (skip v3.5.x)
+- **Type**: Automatic (zero downtime)
+- **Breaking Changes**: None
+- **Rollback**: Requires backup restore
+- **Token Savings**: 60-70% (help queries)
+
+**Always backup before upgrading.**

package/docs/SHARED_CONCEPTS.md

@@ -1,88 +1,43 @@
 # Shared Concepts Reference
 
-> **Single Source of Truth** - This document defines common concepts used across all MCP-SQLEW documentation. Always reference this file for authoritative definitions.
+> **Single Source of Truth** - Common concepts used across all MCP-SQLEW documentation.
 
 ## Table of Contents
 - [Architecture Layers](#architecture-layers)
 - [Enum Values Reference](#enum-values-reference)
-- [Atomic Mode Explained](#atomic-mode-explained)
-- [Action Parameter Requirement](#action-parameter-requirement)
+- [Atomic Mode](#atomic-mode)
+- [Action Parameter](#action-parameter)
 
 ---
 
 ## Architecture Layers
 
-The system uses a 5-layer architecture for organizing decisions, constraints, file changes, and tasks:
+5-layer architecture for organizing decisions, constraints, file changes, and tasks:
 
-### 1. **presentation** - User Interface Layer
-**Definition**: Components that handle user interaction and data presentation.
+### 1. **presentation** - User Interface
+UI components, API endpoints, CLI handlers, forms, response formatting
 
-**Examples**:
-- React/Vue components, UI templates
-- API endpoints (REST/GraphQL controllers)
-- CLI command handlers
-- Web forms and validation logic
-- Response formatting and serialization
+**Examples**: React components, REST controllers, web forms
 
-**When to Use**: Anything users directly interact with or that formats data for external consumption.
+### 2. **business** - Business Logic
+Core application logic, business rules, domain operations
 
----
-
-### 2. **business** - Business Logic Layer
-**Definition**: Core application logic, business rules, and domain operations.
-
-**Examples**:
-- Service classes and business workflows
-- Domain models and entities
-- Validation rules and business constraints
-- Use cases and application services
-- State machines and process orchestration
-
-**When to Use**: Core logic that defines "what" the application does, independent of how it's presented or stored.
-
----
-
-### 3. **data** - Data Access Layer
-**Definition**: Components responsible for data persistence and retrieval.
-
-**Examples**:
-- Database schemas and migrations
-- Repository patterns and ORMs
-- Data access objects (DAOs)
-- Query builders and stored procedures
-- Database connection management
+**Examples**: Service classes, domain models, workflows, validation rules
 
-**When to Use**: Anything that reads from or writes to persistent storage.
+### 3. **data** - Data Access
+Data persistence and retrieval
 
----
-
-### 4. **infrastructure** - Infrastructure Layer
-**Definition**: Technical capabilities and external service integrations.
+**Examples**: Database schemas, repositories, ORMs, queries
 
-**Examples**:
-- Authentication/authorization mechanisms
-- Logging and monitoring systems
-- Message queues and event buses
-- Email/SMS service integrations
-- File storage and CDN integrations
-- Caching mechanisms (Redis, Memcached)
+### 4. **infrastructure** - Infrastructure
+Technical capabilities and external services
 
-**When to Use**: Supporting services that provide technical capabilities to other layers.
-
----
+**Examples**: Auth, logging, message queues, caching, email/SMS services
 
 ### 5. **cross-cutting** - Cross-Cutting Concerns
-**Definition**: Aspects that span multiple layers or affect the entire application.
+Aspects spanning multiple layers
 
-**Examples**:
-- Error handling strategies
-- Security policies and encryption
-- Performance optimization patterns
-- Internationalization (i18n)
-- Audit logging across all layers
-- Configuration management
-
-**When to Use**: Concerns that don't belong to a single layer and affect multiple parts of the system.
+**Examples**: Error handling, security, performance, i18n, configuration
 
 ---
 
@@ -159,181 +114,82 @@ type TaskStatus =
 - `done` → `archived`
 - `archived` → (terminal state, no transitions)
 
-**Auto-Stale Detection**:
+**Auto-Stale Detection & Auto-Archive**:
 - `in_progress` >2 hours → auto-move to `waiting_review`
 - `waiting_review` >24 hours → auto-move to `todo`
+- `done` >48 hours → auto-move to `archived` (weekend-aware)
 
 ---
 
-## Atomic Mode Explained
-
-### What is Atomic Mode?
-
-**Atomic mode** determines how batch operations handle failures:
-
-- **`atomic: true`** (All-or-Nothing)
-  - ALL operations succeed, or ALL fail
-  - Uses database transactions
-  - Rollback on any error
-  - Data consistency guaranteed
-
-- **`atomic: false`** (Best-Effort)
-  - Each operation attempted independently
-  - Partial success possible
-  - Failed items reported in response
-  - Maximum throughput
-
-### When to Use Each Mode
-
-#### Use `atomic: true` (Default) When:
-- **Data consistency is critical**
-  - Financial transactions
-  - Multi-step workflows that must complete together
-  - Related records that must all exist or none exist
-
-- **Validation is important**
-  - You want to validate ALL items before committing ANY
-  - One invalid item should prevent all changes
-
-- **Examples**:
-  ```typescript
-  // All 3 decisions must be set together or none at all
-  set_batch({
-    decisions: [
-      { key: "auth_method", value: "jwt", layer: "infrastructure" },
-      { key: "session_timeout", value: "3600", layer: "infrastructure" },
-      { key: "refresh_token_enabled", value: "true", layer: "infrastructure" }
-    ],
-    atomic: true  // If any fails, rollback all
-  })
-  ```
-
-#### Use `atomic: false` When:
-- **Partial success is acceptable**
-  - Bulk imports where some failures are expected
-  - Idempotent operations (safe to retry)
-  - Performance is critical
-
-- **AI agents making best-effort updates**
-  - Don't want one bad item to block all others
-  - Can handle partial success in response
-
-- **Examples**:
-  ```typescript
-  // Try to send all messages, report which ones failed
-  send_batch({
-    messages: [/* 50 messages */],
-    atomic: false  // Send as many as possible
-  })
-  ```
-
-### Batch Operation Support
-
-Currently supported in:
-- `decision` tool: `action: "set_batch"` (atomic parameter available)
-- `message` tool: `action: "send_batch"` (atomic parameter available)
-- `file` tool: `action: "record_batch"` (atomic parameter available)
-- `task` tool: `action: "batch_create"` (atomic parameter available)
-
-### Performance Implications
-
-- **Atomic mode (`atomic: true`)**:
-  - Slower (transaction overhead)
-  - Higher memory usage (holds all changes until commit)
-  - Safer (guaranteed consistency)
-
-- **Non-atomic mode (`atomic: false`)**:
-  - Faster (no transaction overhead)
-  - Lower memory usage (commit per operation)
-  - More flexible (partial success handling)
+## Atomic Mode
 
----
+Determines batch operation failure handling:
 
-## Action Parameter Requirement
+**`atomic: true`** (All-or-Nothing) - Default
+- ALL succeed or ALL fail
+- Database transaction with rollback
+- Guaranteed consistency
 
-### Why is `action` Required?
+**`atomic: false`** (Best-Effort)
+- Independent operations
+- Partial success possible
+- Failed items reported
 
-**All MCP tools in this system use action-based routing**. The `action` parameter is **REQUIRED** in every tool call.
+### When to Use
 
-### Design Rationale
+**Use `atomic: true`** for:
+- Critical data consistency (financial, multi-step workflows)
+- All-or-nothing validation
 
-1. **Token Efficiency** (96% reduction achieved)
-   - Single tool with multiple actions vs. many separate tools
-   - 20 tools → 7 tools (v2.0.0 → v3.0.0)
-   - Tool definitions: 12,848 tokens → 481 tokens
+**Use `atomic: false`** for:
+- Bulk imports with expected failures
+- AI agent best-effort updates
+- Performance-critical operations
 
-2. **Logical Grouping**
-   - Related operations grouped in one tool
-   - `decision` tool handles all decision operations
-   - `message` tool handles all messaging operations
-   - Etc.
+### Supported Tools
+- `decision`: `set_batch`
+- `message`: `send_batch`
+- `file`: `record_batch`
+- `task`: `batch_create`
 
-3. **Discoverability**
-   - Each tool's actions are documented together
-   - `action: "help"` provides on-demand documentation
-   - Reduces upfront token cost (no documentation until requested)
+---
 
-### Common Error
+## Action Parameter
 
-```json
-❌ ERROR: "Unknown action: undefined"
-```
+**`action` parameter is REQUIRED in all tool calls**
 
-**Cause**: Missing `action` parameter
+### Why Required?
+- Token efficiency: 96% reduction (20 tools → 7 tools)
+- Logical grouping of related operations
+- On-demand help via `action: "help"`
 
-**Fix**: Always include `action` as the first parameter:
+### Common Error
 ```json
-✅ CORRECT:
-{
-  "action": "get",
-  "key": "auth_method"
-}
-
-❌ INCORRECT:
-{
-  "key": "auth_method"  // Missing action parameter
-}
-```
-
-### Available Actions by Tool
-
-- **decision**: `set`, `get`, `list`, `search_tags`, `search_layer`, `versions`, `quick_set`, `search_advanced`, `set_batch`, `has_updates`, `set_from_template`, `create_template`, `list_templates`, `hard_delete`, `help`
-- **message**: `send`, `get`, `mark_read`, `send_batch`, `help`
-- **file**: `record`, `get`, `check_lock`, `record_batch`, `help`
-- **constraint**: `add`, `get`, `deactivate`, `help`
-- **stats**: `layer_summary`, `db_stats`, `clear`, `activity_log`, `flush`, `help`
-- **config**: `get`, `update`, `help`
-- **task**: `create`, `update`, `get`, `list`, `move`, `link`, `archive`, `batch_create`, `help`
-
-### Getting Help
+❌ ERROR: "Unknown action: undefined"
 
-Every tool supports `action: "help"` for comprehensive documentation:
-```json
-{
-  "action": "help"
-}
+// Fix: Always include action
+✅ { "action": "get", "key": "auth_method" }
+❌ { "key": "auth_method" }
 ```
 
-This returns detailed usage instructions, parameter requirements, valid values, and examples for that specific tool.
+### Available Actions
+
+- **decision**: set, get, list, search_tags, search_layer, versions, set_batch, help
+- **message**: send, get, mark_read, send_batch, help
+- **file**: record, get, check_lock, record_batch, help
+- **constraint**: add, get, deactivate, help
+- **stats**: layer_summary, db_stats, clear, help
+- **config**: get, update, help
+- **task**: create, update, get, list, move, link, archive, batch_create, help
 
 ---
 
 ## Database Enum Mappings
 
-For reference, enum values are stored as integers in the database:
+Enum values stored as integers (MCP tools auto-convert - use strings in calls):
 
 - **status**: 1=active, 2=deprecated, 3=draft
 - **msg_type**: 1=decision, 2=warning, 3=request, 4=info
 - **priority**: 1=low, 2=medium, 3=high, 4=critical
 - **change_type**: 1=created, 2=modified, 3=deleted
 - **task_status**: 1=todo, 2=in_progress, 3=waiting_review, 4=blocked, 5=done, 6=archived
-
-The MCP tools handle string↔integer conversion automatically. Always use string values in tool calls.
-
----
-
-## Version History
-
-- **v3.0.0**: Added task_status enum and Auto-Stale Detection section
-- **v2.1.0**: Added Atomic Mode Explained section
-- **v2.0.0**: Initial creation with action-based API concepts

package/docs/TASK_ACTIONS.md

@@ -562,6 +562,8 @@ Link task to decision, constraint, or file.
 
 Archive completed task (soft delete).
 
+**Note:** Tasks in `done` status are automatically archived after 48 hours (configurable via `auto_archive_done_days`). Manual archiving is useful for immediate archival needs.
+
 ### Parameters
 
 **Required:**
@@ -570,6 +572,7 @@ Archive completed task (soft delete).
 
 ### Example
 
+**Manual Archive:**
 ```javascript
 {
   action: "archive",
@@ -577,6 +580,15 @@ Archive completed task (soft delete).
 }
 ```
 
+**Auto-Archive Configuration:**
+```javascript
+// Change auto-archive threshold (via config tool)
+{
+  action: "update",
+  auto_archive_done_days: "3"  // Archive after 3 days instead of default 2 days
+}
+```
+
 ### Response (Success)
 
 ```javascript