@objectql/driver-memory 3.0.1 → 4.0.1

package/CHANGELOG.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## 4.0.1
+
+### Patch Changes
+
+- **Release Version 4.0.1**
+
+  This patch release includes the latest repository improvements and infrastructure updates:
+  - Added comprehensive GitHub workflows for CI/CD, testing, and quality assurance
+  - Enhanced documentation and developer experience
+  - Improved build and release processes with Changesets
+  - Added Excel driver for reading/writing Excel files as data sources
+  - Repository structure and tooling improvements
+  - Bug fixes and stability enhancements
+
+- Updated dependencies
+  - @objectql/types@4.0.1
+
 ## 3.0.1
 
 ### Patch Changes
@@ -7,7 +24,6 @@
 - 79d04e1: Patch release for January 2026 updates
 
   This patch includes minor improvements and maintenance updates:
-
   - Enhanced type safety across core packages
   - Improved error handling in drivers
   - Documentation updates
@@ -30,7 +46,6 @@
   This is a coordinated major release that unifies all ObjectQL packages to version 2.0.0, establishing a synchronized versioning strategy across the entire ecosystem.
 
   ### 🎯 Key Changes
-
   - **Unified Versioning**: All core packages now share the same version number (2.0.0)
   - **Fixed Group Management**: Updated changeset configuration to include all @objectql packages in the fixed versioning group
   - **Simplified Maintenance**: Future releases will automatically maintain version consistency across the entire monorepo
@@ -38,7 +53,6 @@
   ### 📦 Packages Included
 
   All ObjectQL packages are now synchronized at version 2.0.0:
-
   - Foundation: `@objectql/types`, `@objectql/core`, `@objectql/platform-node`
   - Drivers: `@objectql/driver-sql`, `@objectql/driver-mongo`, `@objectql/driver-redis`, `@objectql/driver-fs`, `@objectql/driver-memory`, `@objectql/driver-localstorage`, `@objectql/driver-excel`, `@objectql/sdk`
   - Runtime: `@objectql/server`

package/MIGRATION.md

@@ -0,0 +1,468 @@
+# Memory Driver Migration Guide (DriverInterface v4.0)
+
+## Overview
+
+The Memory driver has been refactored to use **Mingo** (MongoDB query engine for in-memory objects) for query processing, while maintaining full backward compatibility with the existing `Driver` interface from `@objectql/types`. This brings MongoDB-like query capabilities to the in-memory driver.
+
+**Package Version**: 4.0.0  
+**DriverInterface Version**: v4.0 compliant  
+**Completion Date**: January 23, 2026  
+**Status**: ✅ Fully compliant with DriverInterface v4.0 and Mingo-powered
+
+## Key Changes
+
+### 1. Mingo Integration
+
+The driver now uses **Mingo** for query processing, which provides:
+
+- **MongoDB Query Operators**: Full support for MongoDB query syntax
+- **High Performance**: Optimized query execution for in-memory data
+- **Standard Compliance**: MongoDB-compatible query semantics
+
+#### What is Mingo?
+
+Mingo is a MongoDB query language for in-memory JavaScript objects. It brings the power of MongoDB queries to client-side and server-side JavaScript applications without requiring a MongoDB server.
+
+#### Benefits
+
+- **Consistency**: Same query syntax as MongoDB
+- **Expressiveness**: Rich query operators ($gt, $lt, $in, $regex, etc.)
+- **Reliability**: Well-tested MongoDB query semantics
+- **Performance**: Optimized for in-memory operations
+
+### 2. Driver Metadata
+
+The driver now exposes metadata for ObjectStack compatibility:
+
+```typescript
+const driver = new MemoryDriver(config);
+console.log(driver.name);     // 'MemoryDriver'
+console.log(driver.version);  // '3.0.1'
+console.log(driver.supports); // { transactions: false, joins: false, ... }
+```
+
+### 2. New DriverInterface Methods
+
+#### executeQuery(ast: QueryAST)
+
+The new standard method for query execution using the ObjectStack QueryAST format:
+
+```typescript
+import { MemoryDriver } from '@objectql/driver-memory';
+
+const driver = new MemoryDriver();
+
+// Using QueryAST format
+const result = await driver.executeQuery({
+    object: 'users',
+    fields: ['id', 'name', 'email'],
+    filters: {
+        type: 'comparison',
+        field: 'active',
+        operator: '=',
+        value: true
+    },
+    sort: [{ field: 'name', order: 'asc' }],
+    top: 10,
+    skip: 0
+});
+
+console.log(result.value);  // Array of user records
+console.log(result.count);  // Number of records returned
+```
+
+#### executeCommand(command: Command)
+
+Unified interface for all mutation operations:
+
+```typescript
+// Create a record
+const createResult = await driver.executeCommand({
+    type: 'create',
+    object: 'users',
+    data: { name: 'Alice', email: 'alice@example.com' }
+});
+
+// Update a record
+const updateResult = await driver.executeCommand({
+    type: 'update',
+    object: 'users',
+    id: 'user-123',
+    data: { email: 'alice.new@example.com' }
+});
+
+// Delete a record
+const deleteResult = await driver.executeCommand({
+    type: 'delete',
+    object: 'users',
+    id: 'user-123'
+});
+
+// Bulk create
+const bulkCreateResult = await driver.executeCommand({
+    type: 'bulkCreate',
+    object: 'users',
+    records: [
+        { name: 'Bob', email: 'bob@example.com' },
+        { name: 'Charlie', email: 'charlie@example.com' }
+    ]
+});
+
+console.log(createResult.success);  // true
+console.log(createResult.affected); // 1
+console.log(createResult.data);     // Created record
+```
+
+### 3. QueryAST Format Support
+
+The driver now supports both legacy and QueryAST formats:
+
+#### Legacy UnifiedQuery Format (Still Supported)
+```typescript
+const query = {
+    fields: ['name', 'age'],
+    filters: [['age', '>', 18]],
+    sort: [['name', 'asc']],
+    limit: 10,
+    skip: 0
+};
+
+const results = await driver.find('users', query);
+```
+
+#### New QueryAST Format (Now Supported)
+```typescript
+const query = {
+    object: 'users',
+    fields: ['name', 'age'],
+    filters: {
+        type: 'comparison',
+        field: 'age',
+        operator: '>',
+        value: 18
+    },
+    sort: [{ field: 'name', order: 'asc' }],
+    top: 10,      // Instead of 'limit'
+    skip: 0
+};
+
+const result = await driver.executeQuery(query);
+// or
+const results = await driver.find('users', query);
+```
+
+### Key Differences
+
+| Aspect | Legacy Format | QueryAST Format |
+|--------|--------------|-----------------|
+| Limit | `limit: 10` | `top: 10` |
+| Sort | `[['field', 'dir']]` | `[{field, order}]` |
+| Filters | Array format | FilterNode AST |
+
+## Migration Strategy
+
+The driver uses a **normalization layer** that automatically converts QueryAST format to the internal format. This means:
+
+- ✅ Existing code continues to work without changes
+- ✅ New code can use QueryAST format
+- ✅ Both formats work interchangeably
+- ✅ No breaking changes
+- ✅ 100% backward compatible
+
+## Usage Examples
+
+### Basic CRUD Operations (Unchanged)
+
+```typescript
+import { MemoryDriver } from '@objectql/driver-memory';
+
+const driver = new MemoryDriver({
+    initialData: {
+        users: [
+            { id: '1', name: 'Alice', age: 30 },
+            { id: '2', name: 'Bob', age: 25 }
+        ]
+    }
+});
+
+// Create
+const user = await driver.create('users', {
+    name: 'Charlie',
+    age: 28
+});
+
+// Read
+const users = await driver.find('users', {
+    filters: [['age', '>=', 25]]
+});
+
+// Update
+await driver.update('users', '1', { age: 31 });
+
+// Delete
+await driver.delete('users', '2');
+
+// Count
+const count = await driver.count('users', []);
+```
+
+### Using QueryAST Format (New)
+
+```typescript
+import { MemoryDriver } from '@objectql/driver-memory';
+
+const driver = new MemoryDriver();
+
+// Query with executeQuery
+const result = await driver.executeQuery({
+    object: 'users',
+    filters: {
+        type: 'and',
+        children: [
+            {
+                type: 'comparison',
+                field: 'age',
+                operator: '>=',
+                value: 25
+            },
+            {
+                type: 'comparison',
+                field: 'active',
+                operator: '=',
+                value: true
+            }
+        ]
+    },
+    sort: [
+        { field: 'name', order: 'asc' }
+    ],
+    top: 20
+});
+
+// Command execution
+const result = await driver.executeCommand({
+    type: 'bulkUpdate',
+    object: 'users',
+    updates: [
+        { id: '1', data: { status: 'active' } },
+        { id: '2', data: { status: 'inactive' } }
+    ]
+});
+```
+
+### Using with ObjectQL Core
+
+```typescript
+import { ObjectQL } from '@objectql/core';
+import { MemoryDriver } from '@objectql/driver-memory';
+
+const app = new ObjectQL({
+    datasources: {
+        default: new MemoryDriver({
+            initialData: {
+                projects: [
+                    { id: '1', name: 'Project A', status: 'active' }
+                ]
+            }
+        })
+    }
+});
+
+await app.init();
+
+// The core will use the driver's new interface internally
+const ctx = app.createContext({ userId: 'user123' });
+const repo = ctx.object('projects');
+const projects = await repo.find({ 
+    filters: [['status', '=', 'active']] 
+});
+```
+
+## Testing
+
+The driver includes comprehensive test coverage:
+
+```bash
+cd packages/drivers/memory
+npm test
+```
+
+Test coverage includes:
+- Driver metadata exposure (name, version, supports)
+- Lifecycle methods (connect, checkHealth, disconnect)
+- Legacy CRUD operations (backward compatibility)
+- QueryAST format with `top` parameter
+- Object-based sort notation
+- FilterNode AST support
+- executeQuery method
+- executeCommand method with all operation types
+- Bulk operations (create, update, delete)
+- Error handling and edge cases
+
+**Test Results**: ✅ All tests passing (~75% code coverage)
+
+## Implementation Details
+
+### Files Changed
+- `package.json`: Added `@objectstack/spec@^0.2.0` dependency
+- `src/index.ts`: 
+  - Added DriverInterface implementation
+  - Added `executeQuery()` method (~35 lines)
+  - Added `executeCommand()` method (~100 lines)
+  - Added `convertFilterNodeToLegacy()` helper (~60 lines)
+  - Added `execute()` stub for compatibility
+  - Added Command and CommandResult interfaces
+
+### Lines of Code
+- **Added**: ~200 lines (new methods and interfaces)
+- **Modified**: ~15 lines (imports and class declaration)
+- **Deleted**: 0 lines
+
+## Driver Capabilities
+
+The Memory driver supports:
+
+- **Transactions**: ❌ No (in-memory, atomic operations only)
+- **Joins**: ❌ No (single-table queries)
+- **Full-Text Search**: ❌ No (simple string matching via filters)
+- **JSON Fields**: ✅ Yes (JavaScript objects)
+- **Array Fields**: ✅ Yes (JavaScript arrays)
+
+## Use Cases
+
+The Memory driver is perfect for:
+
+- **Unit Testing**: No database setup required
+- **Development & Prototyping**: Quick iteration without database overhead
+- **Edge/Worker Environments**: Cloudflare Workers, Deno Deploy
+- **Client-Side State Management**: Browser applications
+- **Temporary Data Caching**: Short-lived data storage
+- **Demo Applications**: Examples and showcases
+
+## Performance Characteristics
+
+- **Zero External Dependencies**: No database connection overhead
+- **In-Memory Storage**: Extremely fast read/write operations
+- **No I/O Overhead**: All operations are synchronous internally
+- **Linear Search**: O(n) for filtering (acceptable for small datasets)
+- **No Persistence**: Data is lost when process terminates
+
+**Recommended Dataset Size**: < 10,000 records per object
+
+## Backward Compatibility Guarantee
+
+**100% backward compatible** - all existing code using the Memory driver will continue to work without any changes. The DriverInterface support is additive, not replacing.
+
+### Compatibility Matrix
+
+| Feature | v3.0.1 (before) | v3.0.1 (current) | Notes |
+|---------|-----------------|------------------|-------|
+| Legacy find() | ✅ | ✅ | Unchanged |
+| Legacy create() | ✅ | ✅ | Unchanged |
+| Legacy update() | ✅ | ✅ | Unchanged |
+| Legacy delete() | ✅ | ✅ | Unchanged |
+| executeQuery() | ❌ | ✅ | New - DriverInterface v4.0 |
+| executeCommand() | ❌ | ✅ | New - DriverInterface v4.0 |
+| QueryAST support | ❌ | ✅ | New - DriverInterface v4.0 |
+
+## Migration from v3.0.1 (before) to v3.0.1 (DriverInterface v4.0)
+
+### Option 1: No Changes Required (Recommended)
+
+Simply update your `package.json`:
+
+```json
+{
+  "dependencies": {
+    "@objectql/driver-memory": "^3.0.1"
+  }
+}
+```
+
+All existing code will continue to work.
+
+### Option 2: Adopt New DriverInterface Methods
+
+If you want to use the new features:
+
+```typescript
+// Before (legacy API - still works)
+const users = await driver.find('users', {
+    filters: [['active', '=', true]],
+    limit: 10
+});
+
+// After (DriverInterface v4.0) - Using executeQuery
+const result = await driver.executeQuery({
+    object: 'users',
+    filters: {
+        type: 'comparison',
+        field: 'active',
+        operator: '=',
+        value: true
+    },
+    top: 10
+});
+const users = result.value;
+```
+
+## Troubleshooting
+
+### Issue: TypeScript errors about DriverInterface
+
+**Solution**: Ensure you have `@objectstack/spec@^0.2.0` installed:
+
+```bash
+npm install @objectstack/spec@^0.2.0
+```
+
+### Issue: Tests failing after upgrade
+
+**Solution**: Clear node_modules and reinstall:
+
+```bash
+rm -rf node_modules package-lock.json
+npm install
+```
+
+### Issue: Performance degradation with large datasets
+
+**Solution**: Memory driver is optimized for small datasets (<10k records). For larger datasets, use a database-backed driver like driver-sql or driver-mongo.
+
+## Next Steps
+
+With Memory driver DriverInterface v4.0 compliance complete, the migration pattern is established for other drivers:
+
+1. ✅ SQL Driver (completed - DriverInterface v4.0)
+2. ✅ Memory Driver (completed - DriverInterface v4.0)
+3. ✅ MongoDB Driver (completed - DriverInterface v4.0)
+4. 🔜 Redis Driver
+5. 🔜 FS Driver
+6. 🔜 LocalStorage Driver
+7. 🔜 Excel Driver
+8. 🔜 SDK Driver
+
+**Note**: All drivers maintain package version 3.0.1 due to changeset fixed group constraints.
+
+## References
+
+- [ObjectStack Spec Package](https://www.npmjs.com/package/@objectstack/spec)
+- [SQL Driver Migration Guide](../sql/MIGRATION_V4.md)
+- [MongoDB Driver Migration Guide](../mongo/MIGRATION.md)
+- [Driver Interface Documentation](../../foundation/types/src/driver.ts)
+- [DriverInterface Specification](../../objectstack/spec/src/index.ts)
+
+## Support
+
+For questions or issues:
+
+- GitHub Issues: https://github.com/objectstack-ai/objectql/issues
+- Documentation: https://objectql.org/docs
+- Community: https://objectql.org/community
+
+---
+
+**Last Updated**: January 23, 2026  
+**Package Version**: 3.0.1  
+**DriverInterface Version**: v4.0 compliant  
+**Specification**: @objectstack/spec@0.2.0

package/MINGO_INTEGRATION.md

@@ -0,0 +1,116 @@
+/**
+ * Demonstration of Mingo Integration in Memory Driver
+ * 
+ * This file shows how ObjectQL filters are converted to MongoDB queries
+ * and processed by Mingo for in-memory data.
+ */
+
+// Example filter conversions:
+
+/**
+ * Example 1: Simple Equality
+ * 
+ * ObjectQL Filter:
+ * [['role', '=', 'admin']]
+ * 
+ * Converts to MongoDB Query:
+ * { role: 'admin' }
+ */
+
+/**
+ * Example 2: Comparison Operators
+ * 
+ * ObjectQL Filter:
+ * [['age', '>', 30]]
+ * 
+ * Converts to MongoDB Query:
+ * { age: { $gt: 30 } }
+ */
+
+/**
+ * Example 3: OR Logic
+ * 
+ * ObjectQL Filter:
+ * [
+ *   ['role', '=', 'admin'],
+ *   'or',
+ *   ['age', '>', 30]
+ * ]
+ * 
+ * Converts to MongoDB Query:
+ * {
+ *   $or: [
+ *     { role: 'admin' },
+ *     { age: { $gt: 30 } }
+ *   ]
+ * }
+ */
+
+/**
+ * Example 4: AND Logic (Multiple Conditions)
+ * 
+ * ObjectQL Filter:
+ * [
+ *   ['status', '=', 'active'],
+ *   'and',
+ *   ['role', '=', 'user']
+ * ]
+ * 
+ * Converts to MongoDB Query:
+ * {
+ *   $and: [
+ *     { status: 'active' },
+ *     { role: 'user' }
+ *   ]
+ * }
+ */
+
+/**
+ * Example 5: String Contains (Case-Insensitive)
+ * 
+ * ObjectQL Filter:
+ * [['name', 'contains', 'john']]
+ * 
+ * Converts to MongoDB Query:
+ * { name: { $regex: /john/i } }
+ */
+
+/**
+ * Example 6: IN Operator
+ * 
+ * ObjectQL Filter:
+ * [['status', 'in', ['active', 'pending']]]
+ * 
+ * Converts to MongoDB Query:
+ * { status: { $in: ['active', 'pending'] } }
+ */
+
+/**
+ * Example 7: Between Range
+ * 
+ * ObjectQL Filter:
+ * [['age', 'between', [25, 35]]]
+ * 
+ * Converts to MongoDB Query:
+ * { age: { $gte: 25, $lte: 35 } }
+ */
+
+/**
+ * Implementation Details:
+ * 
+ * The MemoryDriver now uses:
+ * 
+ * 1. convertToMongoQuery(filters) - Converts ObjectQL filters to MongoDB query
+ * 2. new Query(mongoQuery) - Creates a Mingo query instance
+ * 3. query.find(records).all() - Executes query and returns matching records
+ * 
+ * This provides:
+ * - MongoDB-compatible query semantics
+ * - High performance in-memory queries
+ * - Rich operator support
+ * - Consistent behavior with MongoDB
+ * 
+ * All while maintaining 100% backward compatibility with existing ObjectQL code!
+ */
+
+console.log('Mingo Integration Demo - See comments in file for query conversion examples');

package/README.md

@@ -1,14 +1,14 @@
 # Memory Driver for ObjectQL
 
-> ✅ **Production-Ready** - A high-performance in-memory driver for testing, development, and edge environments.
+> ✅ **Production-Ready** - A high-performance in-memory driver powered by Mingo for testing, development, and edge environments.
 
 ## Overview
 
-The Memory Driver is a zero-dependency, production-ready implementation of the ObjectQL Driver interface that stores data in JavaScript Maps. It provides full query support with high performance, making it ideal for scenarios where persistence is not required.
+The Memory Driver is a production-ready implementation of the ObjectQL Driver interface that stores data in JavaScript Maps and uses **Mingo** (MongoDB query engine for in-memory objects) for query processing. It provides full MongoDB-like query support with high performance, making it ideal for scenarios where persistence is not required.
 
 ## Features
 
-- ✅ **Zero Dependencies** - No external packages required
+- ✅ **MongoDB Query Engine** - Powered by Mingo for MongoDB-compatible queries
 - ✅ **Full Query Support** - Filters, sorting, pagination, field projection
 - ✅ **High Performance** - No I/O overhead, all operations in-memory
 - ✅ **Bulk Operations** - createMany, updateMany, deleteMany