memory-lucia 2.0.1 → 2.0.2

package/CLAWHUB-FIXES.md

@@ -0,0 +1,127 @@
+# ClawHub Audit Fixes
+
+This document summarizes the fixes applied to address ClawHub security audit feedback.
+
+## Issues Fixed
+
+### ✅ 1. Missing Files
+
+| File | Status | Description |
+|------|--------|-------------|
+| `database/schema.sql` | ✅ Created | Complete database schema with tables, indexes, and views |
+| `migrations/v1-to-v2.js` | ✅ Created | Migration script from V1 to V2 database |
+| `references/API.md` | ✅ Created | Complete API documentation |
+
+### ✅ 2. Version Consistency
+
+| Location | Before | After |
+|----------|--------|-------|
+| SKILL.md | 2.0.0 | 2.0.1 |
+| README.md | 2.0.0 | 2.0.1 |
+| package.json | 2.0.1 | 2.0.1 (already correct) |
+
+### ✅ 3. Removed Development Files
+
+Removed from repository:
+- `publish.bat`
+- `publish-with-otp.bat`
+- `push-final.bat`
+- `push-gh-cli.bat`
+- `push-manual.bat`
+- `push-to-github.bat`
+- `push-with-gh.bat`
+
+Updated `.gitignore` to exclude:
+- `*.bat` files
+- Publish documentation files
+- GitHub workflows (if not ready)
+
+### ✅ 4. Source Verification
+
+All source URLs are valid:
+- **npm**: https://www.npmjs.com/package/memory-lucia
+- **GitHub**: https://github.com/wen521/memory-lucia-
+- **Issues**: https://github.com/wen521/memory-lucia-/issues
+
+## Security Considerations Addressed
+
+### Database Safety
+- ✅ Schema file (`schema.sql`) now included with proper table definitions
+- ✅ All views (`v_pending_decisions`, `v_skill_summary`, `v_weekly_learning_report`, `v_high_priority`) defined in schema
+- ✅ Database initialization script (`database/init.js`) properly references schema.sql
+- ✅ Migration script (`migrations/v1-to-v2.js`) included for data portability
+
+### File System Operations
+- ✅ Database path is configurable (default: `./memory-v2.db`)
+- ✅ Backup directory created relative to working directory
+- ✅ No system files or critical directories accessed
+
+### Backup Management
+- ✅ Backup retention configurable via `keepCount` parameter
+- ✅ Automatic cleanup only removes old backups, never active database
+- ✅ Rollback requires explicit backup path selection
+
+## File Structure
+
+```
+memory-v2-skill/
+├── SKILL.md                    # Skill description (version 2.0.1)
+├── README.md                   # Documentation (version 2.0.1)
+├── package.json                # Package metadata
+├── LICENSE                     # MIT License
+├── .gitignore                  # Excludes dev files
+├── api/
+│   └── index.js               # Main API module
+├── database/
+│   ├── init.js                # Database initialization
+│   └── schema.sql             # ✅ Database schema (NEW)
+├── modules/
+│   ├── priority.js            # Priority analysis
+│   ├── learning.js            # Learning tracking
+│   ├── decision.js            # Decision recording
+│   ├── evolution.js           # Skill evolution
+│   └── version.js             # Version management
+├── migrations/
+│   └── v1-to-v2.js            # ✅ Migration script (NEW)
+├── references/
+│   └── API.md                 # ✅ API documentation (NEW)
+└── scripts/
+    └── init-memory.js         # Setup script
+```
+
+## Pre-Publish Checklist
+
+Before publishing to ClawHub/npm:
+
+- [ ] Verify all files listed above are present
+- [ ] Run `npm test` to ensure tests pass
+- [ ] Run `node scripts/init-memory.js` to verify database initialization
+- [ ] Verify no `.bat` files in the package
+- [ ] Verify version is consistent across all files (2.0.1)
+- [ ] Verify GitHub repository is public and accessible
+- [ ] Verify npm package is published and accessible
+
+## Testing
+
+```bash
+# Install dependencies
+npm install
+
+# Initialize database
+node scripts/init-memory.js
+
+# Run tests
+npm test
+```
+
+## Notes for Reviewers
+
+1. **Database Views**: All SQL views referenced in the code are defined in `database/schema.sql`
+2. **Migration**: The `migrations/v1-to-v2.js` script handles data migration from V1 format
+3. **API Documentation**: Complete API reference available in `references/API.md`
+4. **No Executables**: All `.bat` files have been removed; only Node.js scripts remain
+
+---
+
+Last updated: 2026-03-25
+Version: 2.0.1

package/README.md

@@ -104,7 +104,7 @@ SQLite backend with tables:
 
 ## 📋 Version
 
-Current: 2.0.0
+Current: 2.0.2
 
 ## 📄 License
 

package/SKILL.md

@@ -113,7 +113,7 @@ node migrations/v1-to-v2.js old-memory.db
 
 ## Version
 
-Current: 2.0.0
+Current: 2.0.2
 
 ## License
 

package/database/schema.sql

@@ -0,0 +1,152 @@
+-- Memory V2.0 Database Schema
+-- SQLite database schema for OpenClaw Memory System
+
+-- Priority Analysis Table
+CREATE TABLE IF NOT EXISTS memory_priorities (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    msg_id TEXT NOT NULL,
+    conv_id TEXT,
+    priority_level TEXT CHECK(priority_level IN ('critical', 'high', 'medium', 'low')),
+    reasoning TEXT,
+    category TEXT,
+    created_at DATETIME DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Learning Tracking Table
+CREATE TABLE IF NOT EXISTS memory_learning (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    msg_id TEXT NOT NULL,
+    conv_id TEXT,
+    topic TEXT,
+    description TEXT,
+    status TEXT CHECK(status IN ('active', 'paused', 'completed', 'abandoned')) DEFAULT 'active',
+    progress INTEGER DEFAULT 0 CHECK(progress >= 0 AND progress <= 100),
+    started_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+    updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+    completed_at DATETIME
+);
+
+-- Learning Milestones Table
+CREATE TABLE IF NOT EXISTS memory_learning_milestones (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    learning_id INTEGER NOT NULL,
+    title TEXT NOT NULL,
+    description TEXT,
+    achieved_at DATETIME DEFAULT CURRENT_TIMESTAMP,
+    FOREIGN KEY (learning_id) REFERENCES memory_learning(id) ON DELETE CASCADE
+);
+
+-- Decision Records Table
+CREATE TABLE IF NOT EXISTS memory_decisions (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    msg_id TEXT NOT NULL,
+    conv_id TEXT,
+    summary TEXT NOT NULL,
+    context TEXT,
+    expected_outcome TEXT,
+    actual_outcome TEXT,
+    status TEXT CHECK(status IN ('pending', 'implemented', 'validated', 'rejected')) DEFAULT 'pending',
+    review_scheduled_at DATETIME,
+    reviewed_at DATETIME,
+    created_at DATETIME DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Skill Evolution Table
+CREATE TABLE IF NOT EXISTS memory_evolution (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    skill_name TEXT NOT NULL,
+    category TEXT,
+    usage_count INTEGER DEFAULT 0,
+    success_count INTEGER DEFAULT 0,
+    last_used_at DATETIME,
+    first_used_at DATETIME DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Database Version Tracking
+CREATE TABLE IF NOT EXISTS memory_schema_version (
+    version TEXT PRIMARY KEY,
+    applied_at DATETIME DEFAULT CURRENT_TIMESTAMP
+);
+
+-- Insert initial version
+INSERT OR IGNORE INTO memory_schema_version (version) VALUES ('2.0.0');
+
+-- Create indexes for performance
+CREATE INDEX IF NOT EXISTS idx_priorities_level ON memory_priorities(priority_level);
+CREATE INDEX IF NOT EXISTS idx_priorities_created ON memory_priorities(created_at);
+CREATE INDEX IF NOT EXISTS idx_learning_status ON memory_learning(status);
+CREATE INDEX IF NOT EXISTS idx_learning_topic ON memory_learning(topic);
+CREATE INDEX IF NOT EXISTS idx_decisions_status ON memory_decisions(status);
+CREATE INDEX IF NOT EXISTS idx_decisions_review ON memory_decisions(review_scheduled_at);
+CREATE INDEX IF NOT EXISTS idx_evolution_skill ON memory_evolution(skill_name);
+CREATE INDEX IF NOT EXISTS idx_evolution_category ON memory_evolution(category);
+
+-- Views for common queries
+
+-- View: Pending Decisions
+CREATE VIEW IF NOT EXISTS v_pending_decisions AS
+SELECT 
+    d.*,
+    CASE 
+        WHEN review_scheduled_at < CURRENT_TIMESTAMP THEN 'overdue'
+        WHEN review_scheduled_at <= datetime('now', '+7 days') THEN 'due_soon'
+        ELSE 'scheduled'
+    END as review_status
+FROM memory_decisions d
+WHERE status IN ('pending', 'implemented')
+    AND (review_scheduled_at IS NULL OR review_scheduled_at <= datetime('now', '+7 days'))
+ORDER BY review_scheduled_at ASC;
+
+-- View: Skill Summary
+CREATE VIEW IF NOT EXISTS v_skill_summary AS
+SELECT 
+    skill_name,
+    category,
+    usage_count,
+    success_count,
+    CASE 
+        WHEN usage_count > 0 THEN ROUND(100.0 * success_count / usage_count, 2)
+        ELSE 0
+    END as success_rate,
+    last_used_at,
+    first_used_at
+FROM memory_evolution
+ORDER BY usage_count DESC;
+
+-- View: Weekly Learning Report
+CREATE VIEW IF NOT EXISTS v_weekly_learning_report AS
+SELECT 
+    topic,
+    COUNT(*) as session_count,
+    AVG(progress) as avg_progress,
+    MAX(updated_at) as last_activity,
+    SUM(CASE WHEN status = 'completed' THEN 1 ELSE 0 END) as completed_count
+FROM memory_learning
+WHERE updated_at >= datetime('now', '-7 days')
+GROUP BY topic
+ORDER BY last_activity DESC;
+
+-- View: High Priority Items
+CREATE VIEW IF NOT EXISTS v_high_priority AS
+SELECT 
+    'priority' as type,
+    id,
+    msg_id,
+    priority_level as level,
+    category,
+    reasoning as details,
+    created_at
+FROM memory_priorities
+WHERE priority_level IN ('critical', 'high')
+UNION ALL
+SELECT 
+    'decision' as type,
+    id,
+    msg_id,
+    status as level,
+    'decision' as category,
+    summary as details,
+    created_at
+FROM memory_decisions
+WHERE status = 'pending' AND review_scheduled_at <= CURRENT_TIMESTAMP
+ORDER BY created_at DESC;

package/migrations/v1-to-v2.js

@@ -0,0 +1,213 @@
+#!/usr/bin/env node
+/**
+ * Memory V1 to V2 Migration Script
+ * Migrates data from old memory format to V2 schema
+ */
+
+const sqlite3 = require('sqlite3').verbose();
+const fs = require('fs');
+const path = require('path');
+
+class MemoryMigration {
+    constructor(sourceDbPath, targetDbPath = './memory-v2.db') {
+        this.sourceDbPath = sourceDbPath;
+        this.targetDbPath = targetDbPath;
+        this.sourceDb = null;
+        this.targetDb = null;
+    }
+
+    async init() {
+        // Check if source exists
+        if (!fs.existsSync(this.sourceDbPath)) {
+            throw new Error(`Source database not found: ${this.sourceDbPath}`);
+        }
+
+        // Open source database
+        this.sourceDb = new sqlite3.Database(this.sourceDbPath, sqlite3.OPEN_READONLY);
+        
+        // Open/create target database
+        this.targetDb = new sqlite3.Database(this.targetDbPath);
+        
+        console.log('✅ Connected to source and target databases');
+    }
+
+    async runSchema() {
+        const schemaPath = path.join(__dirname, '..', 'database', 'schema.sql');
+        
+        if (!fs.existsSync(schemaPath)) {
+            throw new Error(`Schema file not found: ${schemaPath}`);
+        }
+
+        const schema = fs.readFileSync(schemaPath, 'utf8');
+        const statements = schema
+            .split(';')
+            .map(s => s.trim())
+            .filter(s => s.length > 0);
+
+        for (const statement of statements) {
+            await this.runTarget(statement);
+        }
+        
+        console.log('✅ Target database schema initialized');
+    }
+
+    async migratePriorities() {
+        console.log('🔄 Migrating priorities...');
+        
+        try {
+            const rows = await this.allSource(
+                "SELECT * FROM memory_priorities WHERE created_at >= datetime('now', '-90 days')"
+            );
+
+            for (const row of rows) {
+                await this.runTarget(
+                    `INSERT INTO memory_priorities 
+                     (msg_id, conv_id, priority_level, reasoning, category, created_at)
+                     VALUES (?, ?, ?, ?, ?, ?)`,
+                    [row.msg_id, row.conv_id, row.priority_level, row.reasoning, row.category, row.created_at]
+                );
+            }
+
+            console.log(`✅ Migrated ${rows.length} priority records`);
+        } catch (err) {
+            console.log('ℹ️ No priorities to migrate or table does not exist');
+        }
+    }
+
+    async migrateLearning() {
+        console.log('🔄 Migrating learning records...');
+        
+        try {
+            const rows = await this.allSource(
+                "SELECT * FROM memory_learning WHERE status != 'abandoned'"
+            );
+
+            for (const row of rows) {
+                await this.runTarget(
+                    `INSERT INTO memory_learning 
+                     (msg_id, conv_id, topic, description, status, progress, started_at, updated_at, completed_at)
+                     VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`,
+                    [row.msg_id, row.conv_id, row.topic, row.description, row.status, 
+                     row.progress, row.started_at, row.updated_at, row.completed_at]
+                );
+            }
+
+            console.log(`✅ Migrated ${rows.length} learning records`);
+        } catch (err) {
+            console.log('ℹ️ No learning records to migrate or table does not exist');
+        }
+    }
+
+    async migrateDecisions() {
+        console.log('🔄 Migrating decisions...');
+        
+        try {
+            const rows = await this.allSource(
+                "SELECT * FROM memory_decisions WHERE status IN ('pending', 'implemented')"
+            );
+
+            for (const row of rows) {
+                await this.runTarget(
+                    `INSERT INTO memory_decisions 
+                     (msg_id, conv_id, summary, context, expected_outcome, actual_outcome, 
+                      status, review_scheduled_at, reviewed_at, created_at)
+                     VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
+                    [row.msg_id, row.conv_id, row.summary, row.context, row.expected_outcome,
+                     row.actual_outcome, row.status, row.review_scheduled_at, row.reviewed_at, row.created_at]
+                );
+            }
+
+            console.log(`✅ Migrated ${rows.length} decision records`);
+        } catch (err) {
+            console.log('ℹ️ No decisions to migrate or table does not exist');
+        }
+    }
+
+    async migrateEvolution() {
+        console.log('🔄 Migrating skill evolution...');
+        
+        try {
+            const rows = await this.allSource(
+                "SELECT * FROM memory_evolution WHERE usage_count > 0"
+            );
+
+            for (const row of rows) {
+                await this.runTarget(
+                    `INSERT INTO memory_evolution 
+                     (skill_name, category, usage_count, success_count, last_used_at, first_used_at)
+                     VALUES (?, ?, ?, ?, ?, ?)`,
+                    [row.skill_name, row.category, row.usage_count, row.success_count, 
+                     row.last_used_at, row.first_used_at]
+                );
+            }
+
+            console.log(`✅ Migrated ${rows.length} skill evolution records`);
+        } catch (err) {
+            console.log('ℹ️ No evolution records to migrate or table does not exist');
+        }
+    }
+
+    async runTarget(sql, params = []) {
+        return new Promise((resolve, reject) => {
+            this.targetDb.run(sql, params, function(err) {
+                if (err) reject(err);
+                else resolve({ id: this.lastID, changes: this.changes });
+            });
+        });
+    }
+
+    async allSource(sql, params = []) {
+        return new Promise((resolve, reject) => {
+            this.sourceDb.all(sql, params, (err, rows) => {
+                if (err) reject(err);
+                else resolve(rows);
+            });
+        });
+    }
+
+    async close() {
+        if (this.sourceDb) {
+            await new Promise((resolve) => this.sourceDb.close(resolve));
+        }
+        if (this.targetDb) {
+            await new Promise((resolve) => this.targetDb.close(resolve));
+        }
+        console.log('✅ Database connections closed');
+    }
+
+    async migrate() {
+        try {
+            await this.init();
+            await this.runSchema();
+            await this.migratePriorities();
+            await this.migrateLearning();
+            await this.migrateDecisions();
+            await this.migrateEvolution();
+            
+            console.log('\n🎉 Migration completed successfully!');
+            console.log(`📁 New database: ${path.resolve(this.targetDbPath)}`);
+        } catch (err) {
+            console.error('\n❌ Migration failed:', err.message);
+            process.exit(1);
+        } finally {
+            await this.close();
+        }
+    }
+}
+
+// CLI usage
+if (require.main === module) {
+    const sourceDb = process.argv[2];
+    const targetDb = process.argv[3] || './memory-v2.db';
+
+    if (!sourceDb) {
+        console.log('Usage: node migrations/v1-to-v2.js <source-v1.db> [target-v2.db]');
+        console.log('Example: node migrations/v1-to-v2.js ./memory-v1.db ./memory-v2.db');
+        process.exit(1);
+    }
+
+    const migration = new MemoryMigration(sourceDb, targetDb);
+    migration.migrate();
+}
+
+module.exports = MemoryMigration;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "memory-lucia",
-  "version": "2.0.1",
+  "version": "2.0.2",
   "description": "Advanced memory system for OpenClaw agents with priority analysis, learning tracking, decision recording, and skill evolution",
   "main": "api/index.js",
   "scripts": {