sqlew 3.8.1 → 3.9.0

package/docs/MIGRATION_GUIDE_V3.9.0.md

@@ -0,0 +1,371 @@
+# Migration Guide: v3.9.0
+
+**Release Date:** 2025-11-14
+**Status:** Production Ready
+**Compatibility:** Fully backward compatible with v3.8.x
+
+---
+
+## Overview
+
+v3.9.0 introduces the **Decision Intelligence System** with the new `suggest` tool, along with critical bug fixes for cross-database compatibility and improved test organization.
+
+### Quick Summary
+
+**✅ No Breaking Changes**
+**✅ Automatic Migration**
+**✅ Zero Downtime**
+**🆕 New suggest Tool**
+**🐛 PostgreSQL/MySQL Fixes**
+**🧪 Better Test Organization**
+
+---
+
+## What's New
+
+### 1. Decision Intelligence System (suggest Tool)
+
+**Purpose:** Intelligent decision discovery, duplicate prevention, and consistency maintenance.
+
+**New Actions:**
+```typescript
+// Find decisions by key pattern
+suggest({ action: "by_key", key: "api/*/latency", limit: 5, min_score: 30 })
+
+// Find decisions by tag similarity
+suggest({ action: "by_tags", tags: ["performance", "api"], limit: 5 })
+
+// Combined search (key + tags + layer)
+suggest({ action: "by_context", key: "api/*", tags: ["performance"], layer: "infrastructure" })
+
+// Check for duplicates before creation
+suggest({ action: "check_duplicate", key: "api/users/get/latency", tags: ["api", "performance"] })
+```
+
+**Auto-Trigger Integration:**
+When policies have `suggest_similar=1`, suggestions are automatically triggered and returned in `decision.set` response:
+
+```typescript
+const result = await decision({
+  action: "set",
+  key: "security/cve/2024-001",
+  value: "critical",
+  tags: ["security", "vulnerability"]
+});
+
+// Response includes auto-triggered suggestions
+{
+  success: true,
+  key: "security/cve/2024-001",
+  version: "1.0.0",
+  suggestions: {
+    triggered_by: "security-policy",
+    reason: "Policy has suggest_similar enabled",
+    suggestions: [
+      {
+        key: "security/cve/2023-999",
+        similarity_score: 85,
+        tags: ["security", "vulnerability"],
+        layer: "cross-cutting"
+      }
+    ]
+  }
+}
+```
+
+**Use Cases:**
+- Prevent duplicate decisions
+- Find related decisions for consistency
+- Discover decisions by pattern (e.g., all API latency metrics)
+- Maintain architectural consistency across layers
+
+---
+
+## Migration Steps
+
+### Step 1: Backup Your Database
+
+**IMPORTANT:** Always backup before upgrading.
+
+```bash
+# Backup SQLite database
+cp .sqlew/sqlew.db .sqlew/sqlew.db.backup-$(date +%Y%m%d)
+
+# Backup PostgreSQL (if using)
+pg_dump -h localhost -U sqlew_user sqlew_db > backup-$(date +%Y%m%d).sql
+
+# Backup MySQL (if using)
+mysqldump -h localhost -u sqlew_user -p sqlew_db > backup-$(date +%Y%m%d).sql
+```
+
+### Step 2: Update sqlew
+
+```bash
+# Update via npm
+npm update sqlew
+
+# Or reinstall specific version
+npm install sqlew@3.9.0
+
+# Or use npx (recommended)
+# Update .mcp.json:
+{
+  "mcpServers": {
+    "sqlew": {
+      "command": "npx",
+      "args": ["sqlew@3.9.0"]
+    }
+  }
+}
+```
+
+### Step 3: Restart Claude Code
+
+The server will automatically run migrations on startup.
+
+```bash
+# Restart Claude Code or MCP inspector
+# Migrations run automatically
+```
+
+### Step 4: Verify Migration
+
+Check migration status:
+
+```bash
+# Inspect database
+sqlite3 .sqlew/sqlew.db "SELECT * FROM knex_migrations ORDER BY id DESC LIMIT 5;"
+
+# Expected latest migrations:
+# - 20251114000000_fix_v_tagged_decisions_numeric_support
+# - 20251112000002_fix_task_pruned_files_schema_v3_9_0
+# - 20251112000001_fix_task_file_links_schema_v3_9_0
+```
+
+### Step 5: Test suggest Tool
+
+```bash
+# Test suggest tool
+npx @modelcontextprotocol/inspector npx sqlew
+
+# In inspector:
+suggest({ action: "help" })
+suggest({ action: "by_key", key: "*" })
+```
+
+---
+
+## Bug Fixes Included
+
+### 1. PostgreSQL CAST Type Mismatch (Critical)
+
+**Issue:** PostgreSQL rejected `COALESCE(TEXT, NUMERIC)` in `v_tagged_decisions` view.
+
+**Fixed:**
+- Migration adds database-specific CAST syntax
+- SQL dump export converts views correctly
+- All 20 cross-database tests passing
+
+**Impact:** If you use PostgreSQL or export to PostgreSQL, this fixes import errors.
+
+### 2. FK Constraint Cleanup Order
+
+**Issue:** E2E tests failed with FK constraint violations during cleanup.
+
+**Fixed:**
+- Reordered cleanup to delete child records first
+- All 3 e2e workflow tests passing
+
+**Impact:** Better test reliability, no functional changes for users.
+
+### 3. Task File Links Schema
+
+**Issue:** UNIQUE constraint incorrectly configured on `t_task_file_links`.
+
+**Fixed:**
+- Migration `20251112000001_fix_task_file_links_schema_v3_9_0.ts`
+- Idempotent with existence checks
+
+**Impact:** Better data integrity for task file tracking.
+
+---
+
+## Test Organization Changes
+
+### Docker Test Separation
+
+**Old Structure:**
+```
+src/tests/database/
+  ├── sql-dump/
+  │   ├── cross-database.test.ts  # Required Docker
+  │   └── ...
+  └── migrations/
+      └── schema-migration.test.ts  # Required Docker
+```
+
+**New Structure:**
+```
+src/tests/
+  ├── docker/  # NEW - Docker-dependent tests
+  │   ├── cross-database.test.ts
+  │   ├── schema-migration.test.ts
+  │   └── ... (7 tests total)
+  └── database/  # Unit tests only (no Docker)
+```
+
+**Commands:**
+```bash
+# Unit tests only (no Docker needed)
+npm test  # 481/481 tests, 0 failures
+
+# Docker tests (requires containers)
+npm run test:docker  # 7 test suites
+```
+
+**Impact:** Faster CI, clearer test failures, no Docker dependency for `npm test`.
+
+---
+
+## Git Hook Enhancement
+
+### Pre-Commit Migration Check
+
+**Old Behavior:** Prevented editing any committed migration files.
+
+**New Behavior:** Prevents editing **PUSHED** migration files only.
+
+**Benefits:**
+- ✅ Edit locally committed migrations (not yet pushed)
+- ✅ Prevents editing pushed migrations (already deployed)
+- ✅ Auto-detects remote branch (origin/main, origin/master, origin/dev)
+- ✅ Graceful fallback for local-only repos
+
+**Example:**
+```bash
+# ✅ ALLOWED - Edit local migration (not pushed)
+git commit -m "Fix migration before push"
+
+# ❌ BLOCKED - Edit pushed migration
+# Error: PUSHED migration file was edited: src/config/knex/enhancements/20251114000000_fix_v_tagged_decisions_numeric_support.ts
+```
+
+---
+
+## Rollback Procedure
+
+If you encounter issues:
+
+### Option 1: Restore Database Backup
+
+```bash
+# SQLite
+cp .sqlew/sqlew.db.backup-YYYYMMDD .sqlew/sqlew.db
+
+# PostgreSQL
+psql -h localhost -U sqlew_user sqlew_db < backup-YYYYMMDD.sql
+
+# MySQL
+mysql -h localhost -u sqlew_user -p sqlew_db < backup-YYYYMMDD.sql
+```
+
+### Option 2: Downgrade to v3.8.1
+
+```bash
+# Restore backup first (see above)
+
+# Then downgrade
+npm install sqlew@3.8.1
+
+# Or update .mcp.json
+{
+  "mcpServers": {
+    "sqlew": {
+      "command": "npx",
+      "args": ["sqlew@3.8.1"]
+    }
+  }
+}
+```
+
+---
+
+## Known Issues
+
+**None reported as of v3.9.0 release.**
+
+---
+
+## FAQ
+
+### Q: Do I need to update my code?
+
+**A:** No. v3.9.0 is fully backward compatible. The `suggest` tool is optional.
+
+### Q: Will my existing decisions/tasks work?
+
+**A:** Yes. All existing data is preserved and migrations are automatic.
+
+### Q: How do I use the suggest tool?
+
+**A:** Use `suggest({ action: "help" })` for documentation. See examples above.
+
+### Q: Do I need Docker for testing?
+
+**A:** No. `npm test` runs 481 unit tests without Docker. Docker tests are optional: `npm run test:docker`.
+
+### Q: Can I edit migrations locally?
+
+**A:** Yes, if not pushed yet. The pre-commit hook now allows editing local migrations.
+
+### Q: What if migration fails?
+
+**A:** Restore backup (see Rollback Procedure above) and report issue on GitHub.
+
+---
+
+## Performance Notes
+
+### Code Reduction
+
+- **Deleted:** `src/utils/sql-dump.ts` (-1,799 lines)
+- **Added:** Suggest tool system (+765 lines)
+- **Net:** -239 lines (1.1% code reduction)
+
+### Test Coverage
+
+- **Unit Tests:** 481 tests, 64.85% coverage
+- **E2E Tests:** 3 workflows, 100% passing
+- **Cross-DB Tests:** 20 tests, 100% passing (MySQL, MariaDB, PostgreSQL)
+
+---
+
+## Support
+
+**Issues:** https://github.com/anthropics/sqlew/issues
+**Documentation:** See CHANGELOG.md, CLAUDE.md, and docs/
+**Migration Assistance:** Open GitHub issue with error logs
+
+---
+
+## Verification Checklist
+
+After migration, verify:
+
+- [ ] Server starts without errors
+- [ ] Existing decisions/tasks queryable
+- [ ] suggest tool available: `suggest({ action: "help" })`
+- [ ] Unit tests pass: `npm test` (if running locally)
+- [ ] Database migrations applied: Check `knex_migrations` table
+- [ ] No error logs in `.sqlew/logs/` (if logging enabled)
+
+---
+
+## Next Steps
+
+1. **Explore suggest tool** - Try pattern searches and duplicate detection
+2. **Enable auto-trigger** - Add `suggest_similar=1` to policies
+3. **Update documentation** - Document decisions using suggest tool
+4. **Run Docker tests** - Verify cross-database compatibility (optional)
+
+**Welcome to v3.9.0! 🎉**