@toonstore/torm 0.3.0 → 0.4.0

package/README.md

@@ -50,20 +50,28 @@ pnpm add @toonstore/torm
 # Default: localhost:6379
 
 # Or using Docker
-docker run -d -p 6379:6379 toonstoredb/toonstoredb:latest
+docker run -d -p 6379:6379 samso9th/toonstore:latest
 
 # Or install from releases
-# Download from: https://github.com/toonstore/toonstoredb/releases
+# Download from: https://github.com/kalama-tech/toonstoredb/releases
 ```
 
-> **Note:** ToonStoreDB is a high-performance database with Redis-compatible protocol. The SDK connects directly to your ToonStoreDB instance. No additional servers or middleware needed.
+> **Note:** ToonStoreDB is a high-performance database with Redis-compatible protocol. The SDK connects directly to your ToonStoreDB instance.
 
-### Basic Usage
+### Installation
+
+```bash
+npm install @toonstore/torm
+```
+
+### Step 1: Connect to ToonStoreDB
+
+Create a new file (e.g., `app.ts`):
 
 ```typescript
 import { TormClient } from '@toonstore/torm';
 
-// 1. Connect to ToonStoreDB
+// Connect to ToonStoreDB
 const torm = new TormClient({
   host: 'localhost',
   port: 6379,
@@ -71,7 +79,17 @@ const torm = new TormClient({
   // url: 'toonstoredb://localhost:6379'
 });
 
-// 2. Define your model interface
+// Check connection
+const health = await torm.health();
+console.log('Database health:', health);
+```
+
+### Step 2: Define Your Models
+
+Define TypeScript interfaces and create models with validation:
+
+```typescript
+// Define your data structure
 interface User {
   name: string;
   email: string;
@@ -79,7 +97,7 @@ interface User {
   website?: string;
 }
 
-// 3. Create model with validation schema
+// Create model with schema validation
 const User = torm.model<User>('User', {
   name: {
     type: 'string',
@@ -103,44 +121,79 @@ const User = torm.model<User>('User', {
     url: true,
   },
 });
+```
+
+### Step 3: Perform CRUD Operations
 
-// 4. Create document (ID auto-generated)
+```typescript
+// Create a user (ID auto-generated)
 const user = await User.create({
   name: 'Alice',
   email: 'alice@example.com',
   age: 30,
   website: 'https://alice.dev',
 });
-console.log(user._id); // Auto-generated: "1736076000000-abc123xyz"
+console.log('Created user:', user._id);
 
-// 5. Find documents
+// Find all users
 const allUsers = await User.find();
-const specificUser = await User.findById(user._id);
-const foundUser = await User.findOne({ email: 'alice@example.com' });
 
-// 6. Query with filters
+// Find user by ID
+const foundUser = await User.findById(user._id);
+
+// Find one user by criteria
+const alice = await User.findOne({ email: 'alice@example.com' });
+
+// Update user
+const updated = await User.update(user._id, { age: 31 });
+
+// Delete user
+await User.delete(user._id);
+
+// Count users
+const count = await User.count();
+```
+
+### Step 4: Query with Filters
+
+```typescript
+// Find users 18 and older
 const adults = await User.query()
   .where('age', 'gte', 18)
   .sort('name', 'asc')
+  .exec();
+
+// Pagination
+const page1 = await User.query()
+  .sort('_createdAt', 'desc')
   .limit(10)
+  .skip(0)
   .exec();
 
-// 7. Update document
-const updated = await User.update(user._id, { age: 31 });
+// Complex queries
+const result = await User.query()
+  .where('age', 'gte', 18)
+  .where('age', 'lte', 65)
+  .sort('name', 'asc')
+  .limit(20)
+  .exec();
+```
 
-// 8. Delete document
-await User.delete(user._id);
+### Step 5: Launch TORM Studio (Optional)
 
-// 9. Delete many documents
-await User.deleteMany({ age: { $lt: 18 } });
+Visual database manager for browsing and editing data:
 
-// 10. Count documents
-const count = await User.count();
+```bash
+# First time - auto-generates config
+npx torm studio
 
-// 11. Disconnect when done
-await torm.disconnect();
+# Edit torm.config.ts with your database credentials
+# Then run again:
+npx torm studio
 ```
 
+Opens at http://localhost:4983 with a visual interface to browse collections, edit records, and view stats.
+
 ## 🛠️ CLI Commands
 
 TORM includes a powerful CLI for database management and development workflows:
@@ -159,7 +212,12 @@ npx torm studio
 **Access:** http://localhost:4983
 
 **Configuration:**
-Create or edit `torm.config.ts` in your project root:
+
+You can use either TypeScript or JavaScript config files:
+
+**Option 1: TypeScript Config (Recommended)**
+
+Create `torm.config.ts` in your project root:
 ```typescript
 export default {
   dbCredentials: {
@@ -175,6 +233,30 @@ export default {
 };
 ```
 
+For TypeScript config files, install `tsx` (recommended) or `ts-node`:
+```bash
+npm install -D tsx
+# or
+npm install -D ts-node
+```
+
+**Option 2: JavaScript Config**
+
+Create `torm.config.js` in your project root:
+```javascript
+module.exports = {
+  dbCredentials: {
+    host: 'localhost',
+    port: 6379,
+  },
+  studio: {
+    port: 4983,
+  },
+};
+```
+
+No additional dependencies needed for `.js` config files.
+
 **Features:**
 - 📊 Visual database browser - browse collections and documents
 - ✏️ Create, read, update, delete records with UI
@@ -189,19 +271,174 @@ export default {
 npx torm studio --port 8080
 ```
 
-### Migrations (Coming Soon)
+### Migrations
+
+Create and run data-shape migrations for your schemaless database:
 
 ```bash
-# Generate migration from schema changes
-npx torm generate
+# Create a new migration
+npx torm migrate:create add_age_field
 
 # Run pending migrations
-npx torm migrate
+npx torm migrate:up
+
+# Run specific number of migrations
+npx torm migrate:up 1
+
+# Rollback last migration
+npx torm migrate:down
+
+# Rollback multiple migrations
+npx torm migrate:down 2
 
 # Check migration status
 npx torm migrate:status
 ```
 
+**How Migrations Work with ToonStoreDB:**
+
+ToonStoreDB is schemaless at the storage layer, but your application code assumes a particular data shape. Migrations help you evolve that shape over time.
+
+**Example Workflow:**
+
+```typescript
+// Step 1: Create a migration file
+// $ npx torm migrate:create add_age_to_users
+// Creates: migrations/20260105_add_age_to_users.ts
+
+// migrations/20260105_add_age_to_users.ts
+export default {
+  async up(client) {
+    // Add 'age' field to all existing users with a default value
+    const pattern = 'toonstore:User:*';
+    const keys = await client.keys(pattern);
+    
+    for (const key of keys) {
+      const data = JSON.parse(await client.get(key));
+      if (!data.age) {
+        data.age = 18; // Default value
+        await client.set(key, JSON.stringify(data));
+      }
+    }
+    
+    console.log(`Updated ${keys.length} users with age field`);
+  },
+  
+  async down(client) {
+    // Remove 'age' field from all users
+    const pattern = 'toonstore:User:*';
+    const keys = await client.keys(pattern);
+    
+    for (const key of keys) {
+      const data = JSON.parse(await client.get(key));
+      delete data.age;
+      await client.set(key, JSON.stringify(data));
+    }
+    
+    console.log(`Removed age field from ${keys.length} users`);
+  },
+};
+
+// Step 2: Run migration
+// $ npx torm migrate:up
+// Applies: 20260105_add_age_to_users
+```
+
+**Common Migration Patterns:**
+
+```typescript
+// 1. Add field with default value
+export default {
+  async up(client) {
+    const keys = await client.keys('toonstore:User:*');
+    for (const key of keys) {
+      const user = JSON.parse(await client.get(key));
+      user.verified = user.verified ?? false;
+      await client.set(key, JSON.stringify(user));
+    }
+  },
+  async down(client) {
+    const keys = await client.keys('toonstore:User:*');
+    for (const key of keys) {
+      const user = JSON.parse(await client.get(key));
+      delete user.verified;
+      await client.set(key, JSON.stringify(user));
+    }
+  },
+};
+
+// 2. Rename field
+export default {
+  async up(client) {
+    const keys = await client.keys('toonstore:Post:*');
+    for (const key of keys) {
+      const post = JSON.parse(await client.get(key));
+      if (post.author_id) {
+        post.authorId = post.author_id;
+        delete post.author_id;
+        await client.set(key, JSON.stringify(post));
+      }
+    }
+  },
+  async down(client) {
+    const keys = await client.keys('toonstore:Post:*');
+    for (const key of keys) {
+      const post = JSON.parse(await client.get(key));
+      if (post.authorId) {
+        post.author_id = post.authorId;
+        delete post.authorId;
+        await client.set(key, JSON.stringify(post));
+      }
+    }
+  },
+};
+
+// 3. Data transformation
+export default {
+  async up(client) {
+    const keys = await client.keys('toonstore:User:*');
+    for (const key of keys) {
+      const user = JSON.parse(await client.get(key));
+      // Split full name into first and last
+      if (user.name && !user.firstName) {
+        const [firstName, ...lastName] = user.name.split(' ');
+        user.firstName = firstName;
+        user.lastName = lastName.join(' ');
+        delete user.name;
+        await client.set(key, JSON.stringify(user));
+      }
+    }
+  },
+  async down(client) {
+    const keys = await client.keys('toonstore:User:*');
+    for (const key of keys) {
+      const user = JSON.parse(await client.get(key));
+      if (user.firstName) {
+        user.name = `${user.firstName} ${user.lastName || ''}`.trim();
+        delete user.firstName;
+        delete user.lastName;
+        await client.set(key, JSON.stringify(user));
+      }
+    }
+  },
+};
+```
+
+**Migration Tracking:**
+
+TORM tracks which migrations have run using a special key:
+```
+toonstore:_migrations -> ["20260105_add_age_to_users", "20260106_rename_author_field"]
+```
+
+**Best Practices:**
+
+1. **Never modify old migrations** - create new ones to fix issues
+2. **Always provide `down()` for rollback** - test it works
+3. **Use transactions when available** - ensure atomic changes
+4. **Test on copy of production data** - before running in prod
+5. **Version control migrations** - commit with code changes
+
 ### Type Generation (Coming Soon)
 
 ```bash
@@ -212,6 +449,47 @@ npx torm generate
 npx torm generate --watch
 ```
 
+**How Type Generation Will Work:**
+
+TORM will introspect your database and generate TypeScript interfaces:
+
+```bash
+# Scan your database and generate types
+$ npx torm generate
+
+# Output: src/generated/torm-types.ts
+```
+
+**Generated Types:**
+```typescript
+// src/generated/torm-types.ts (auto-generated)
+export interface User {
+  _id: string;
+  name: string;
+  email: string;
+  age: number;
+  website?: string;
+  _createdAt: string;
+  _updatedAt: string;
+}
+
+export interface Post {
+  _id: string;
+  title: string;
+  content: string;
+  authorId: string;
+  published: boolean;
+  _createdAt: string;
+  _updatedAt: string;
+}
+
+// Use generated types
+import { User, Post } from './generated/torm-types';
+
+const User = torm.model<User>('User');
+const Post = torm.model<Post>('Post');
+```
+
 ## 📖 API Documentation
 
 ### TormClient
@@ -560,32 +838,47 @@ Version 0.2.0 introduces breaking changes:
 - **No more HTTP server required** - SDK now connects directly to ToonStoreDB
 - **Auto-generated IDs** - No need to provide `id` field, use `_id` instead
 - **New connection options** - Use `host`, `port`, or `url` for ToonStoreDB connection
+- **TORM Studio bundled** - Visual database manager included (run `npx torm studio`)
 
 ### Migration Steps
 
+**Old (v0.1.x):**
 ```typescript
-// v0.1.x (OLD - required TORM server via HTTP)
+// Required TORM server via HTTP
 const torm = new TormClient({
   baseURL: 'http://localhost:3001',
 });
 
 const user = await User.create({
-  id: 'user:1',
+  id: 'user:1',  // Manual ID
   name: 'Alice',
 });
+```
 
-// v0.2.x (NEW - direct ToonStoreDB connection)
+**New (v0.2.x):**
+```typescript
+// Direct ToonStoreDB connection
 const torm = new TormClient({
   host: 'localhost',
   port: 6379,
 });
 
 const user = await User.create({
-  name: 'Alice', // No id needed
+  name: 'Alice',  // No id needed
 });
-console.log(user._id); // Auto-generated
+console.log(user._id); // Auto-generated: "1736076000000-abc123xyz"
+
+// Launch Studio
+// $ npx torm studio
+// Opens http://localhost:4983
 ```
 
+**Steps:**
+1. Update connection config (remove `baseURL`, add `host`/`port`)
+2. Remove manual `id` fields from `create()` calls
+3. Use `_id` to access document IDs
+4. (Optional) Create `torm.config.ts` for Studio
+
 ## 🧪 Running Examples
 
 ```bash
@@ -616,6 +909,128 @@ npm run dev
 
 # Run tests
 npm test
+
+# Start Studio
+npm run studio
+```
+
+## 🎯 Complete Example
+
+Here's a complete example building a blog application:
+
+```typescript
+import { TormClient } from '@toonstore/torm';
+
+// 1. Connect to database
+const torm = new TormClient({
+  host: 'localhost',
+  port: 6379,
+});
+
+// 2. Define models
+interface User {
+  username: string;
+  email: string;
+  bio?: string;
+}
+
+interface Post {
+  title: string;
+  content: string;
+  authorId: string;
+  published: boolean;
+  tags: string[];
+}
+
+const User = torm.model<User>('User', {
+  username: {
+    type: 'string',
+    required: true,
+    minLength: 3,
+    pattern: /^[a-zA-Z0-9_]+$/,
+  },
+  email: {
+    type: 'string',
+    required: true,
+    email: true,
+  },
+  bio: {
+    type: 'string',
+    maxLength: 500,
+  },
+});
+
+const Post = torm.model<Post>('Post', {
+  title: {
+    type: 'string',
+    required: true,
+    minLength: 5,
+    maxLength: 200,
+  },
+  content: {
+    type: 'string',
+    required: true,
+    minLength: 10,
+  },
+  authorId: {
+    type: 'string',
+    required: true,
+  },
+  published: {
+    type: 'boolean',
+  },
+  tags: {
+    type: 'array',
+  },
+});
+
+// 3. Use the models
+async function main() {
+  // Create a user
+  const user = await User.create({
+    username: 'alice',
+    email: 'alice@example.com',
+    bio: 'Love coding and coffee ☕',
+  });
+
+  // Create posts
+  await Post.create({
+    title: 'Getting Started with ToonStoreDB',
+    content: 'ToonStoreDB is a blazingly fast database...',
+    authorId: user._id,
+    published: true,
+    tags: ['database', 'toonstoredb', 'tutorial'],
+  });
+
+  await Post.create({
+    title: 'Building a Blog with TORM',
+    content: 'TORM makes it easy to build applications...',
+    authorId: user._id,
+    published: false,
+    tags: ['torm', 'tutorial'],
+  });
+
+  // Query published posts
+  const publishedPosts = await Post.query()
+    .where('published', 'eq', true)
+    .where('authorId', 'eq', user._id)
+    .sort('_createdAt', 'desc')
+    .exec();
+
+  console.log('Published posts:', publishedPosts.length);
+
+  // Get user's post count
+  const postCount = await Post.query()
+    .where('authorId', 'eq', user._id)
+    .count();
+
+  console.log(`${user.username} has ${postCount} posts`);
+
+  // Clean up
+  await torm.disconnect();
+}
+
+main().catch(console.error);
 ```
 
 ## 🌟 Why TORM?
@@ -655,10 +1070,14 @@ Your Node.js App → @toonstore/torm SDK → ToonStoreDB
 
 **CLI Tools:**
 - [x] TORM Studio (bundled Node.js server) ✅
-- [ ] Schema migrations (`torm generate`, `torm migrate`)
-- [ ] Type generation from schemas
+- [x] Schema migrations (data-shape migrations for schemaless DB) ✅
+  - [x] `torm migrate:create` - Create migration file
+  - [x] `torm migrate:up` - Run pending migrations
+  - [x] `torm migrate:down` - Rollback last migration
+  - [x] `torm migrate:status` - Check migration status
+- [ ] Type generation from existing data
 - [ ] Database seeding
-- [ ] Schema validation and linting
+- [ ] Data export/import tools
 
 **Studio Enhancements:**
 - [ ] Custom domain (local.torm.studio via DNS) - no hosts file needed
@@ -668,20 +1087,55 @@ Your Node.js App → @toonstore/torm SDK → ToonStoreDB
 - [ ] Schema visualization
 
 **Migration System (Planned):**
+
+Unlike SQL databases, ToonStoreDB is schemaless—but you still need **data-shape migrations** for application-level schema evolution:
+
 ```bash
-# Generate migration from model changes
-torm generate --name add_users_table
+# Create migration for data transformation
+torm migrate:create add_age_field
 
-# Run migrations
-torm migrate
+# Run pending migrations
+torm migrate:up
 
-# Rollback
-torm migrate:rollback
+# Rollback last migration
+torm migrate:down
 
-# Check status
+# Check what's been applied
 torm migrate:status
 ```
 
+**Example Migration:**
+```typescript
+// migrations/20260105_add_age_field.ts
+export default {
+  async up(client) {
+    // Add 'age' to all users with default value
+    const keys = await client.keys('toonstore:User:*');
+    for (const key of keys) {
+      const user = JSON.parse(await client.get(key));
+      user.age = user.age ?? 18; // Default age
+      await client.set(key, JSON.stringify(user));
+    }
+  },
+  async down(client) {
+    // Remove 'age' from all users
+    const keys = await client.keys('toonstore:User:*');
+    for (const key of keys) {
+      const user = JSON.parse(await client.get(key));
+      delete user.age;
+      await client.set(key, JSON.stringify(user));
+    }
+  },
+};
+```
+
+**Common Use Cases:**
+- Add/remove fields with defaults
+- Rename fields (`author_id` → `authorId`)
+- Transform data (split name into first/last)
+- Back-fill missing values
+- Data cleanup and normalization
+
 ## 🤝 Contributing
 
 Contributions are welcome! Please feel free to submit a Pull Request.