@firtoz/drizzle-indexeddb 0.2.0 → 0.3.0

package/CHANGELOG.md

@@ -1,5 +1,27 @@
 # @firtoz/drizzle-indexeddb
 
+## 0.3.0
+
+### Minor Changes
+
+- [`5e854a6`](https://github.com/firtoz/fullstack-toolkit/commit/5e854a62236a811918a47037a59df23329856614) Thanks [@firtoz](https://github.com/firtoz)! - ### Breaking Changes
+
+  - Removed `migrateIndexedDB` and `IndexedDBMigrationConfig` exports - use `migrateIndexedDBWithFunctions` instead
+  - Removed snapshot-based migration system in favor of function-based migrations
+
+  ### New Features
+
+  - Added `drizzle-indexeddb-generate` CLI tool to generate IndexedDB migration functions from Drizzle snapshots
+  - Added `generateIndexedDBMigrations` export for programmatic migration generation
+  - Added `./generate` export path
+
+  ### Migration Guide
+
+  Instead of importing snapshots directly and using `migrateIndexedDB`, you now:
+
+  1. Run `bun drizzle-indexeddb-generate` (or `npx drizzle-indexeddb-generate`) after `drizzle-kit generate`
+  2. Import the generated migrations and use `migrateIndexedDBWithFunctions`
+
 ## 0.2.0
 
 ### Minor Changes
@@ -41,18 +63,6 @@
   - Sync configuration for real-time updates
   - Works seamlessly with React hooks
 
-  ### Snapshot-Based Migration
-
-  **`migrateIndexedDB(dbName, config, debug?)`** - Automatically migrates IndexedDB databases using Drizzle snapshot files:
-
-  - Reads Drizzle journal and snapshot files
-  - Tracks applied migrations in `__drizzle_migrations` store
-  - Creates/updates object stores and indexes based on schema changes
-  - Handles table deletion, index changes, and schema evolution
-  - Incremental migrations - only applies pending changes
-  - Validates primary key structure changes (requires manual migration if keys change)
-  - Performance logging when debug mode is enabled
-
   ### Function-Based Migration
 
   **`migrateIndexedDBWithFunctions(dbName, migrations, debug?)`** - Run migrations using custom migration functions:
@@ -87,19 +97,15 @@
   ## Example
 
   ```typescript
-  import { migrateIndexedDB } from "@firtoz/drizzle-indexeddb";
-  import journal from "./drizzle/journal.json";
-  import * as snapshots from "./drizzle/snapshots";
+  import { migrateIndexedDBWithFunctions } from "@firtoz/drizzle-indexeddb";
+  import migrations from "./drizzle/indexeddb-migrations";
 
-  // Migrate database using Drizzle snapshots
-  const db = await migrateIndexedDB(
+  // Migrate database using function-based migrations
+  const db = await migrateIndexedDBWithFunctions(
     "my-app-db",
-    {
-      journal,
-      snapshots,
-    },
-    true
-  ); // debug mode
+    migrations,
+    true // debug mode
+  );
 
   // Use with TanStack DB
   import { createCollection } from "@tanstack/db";
@@ -136,9 +142,9 @@
 
   ## Migration Workflow
 
-  1. Generate Drizzle snapshots: `drizzle-kit generate`
-  2. Import journal and snapshots
-  3. Call `migrateIndexedDB()` on app startup
+  1. Generate Drizzle migrations: `drizzle-kit generate`
+  2. Generate IndexedDB migrations: `bun drizzle-indexeddb-generate`
+  3. Import migrations and call `migrateIndexedDBWithFunctions()` on app startup
   4. Database automatically updates to latest schema
 
   ## Dependencies

package/README.md

@@ -1,10 +1,10 @@
 # @firtoz/drizzle-indexeddb
 
-TanStack DB collections backed by IndexedDB with automatic migrations powered by Drizzle ORM snapshots. Build reactive, type-safe IndexedDB applications with the power of Drizzle's schema management.
+TanStack DB collections backed by IndexedDB with automatic migrations powered by Drizzle ORM. Build reactive, type-safe IndexedDB applications with the power of Drizzle's schema management.
 
 > **⚠️ Early WIP Notice:** This package is in very early development and is **not production-ready**. It is TypeScript-only and may have breaking changes. While I (the maintainer) have limited time, I'm open to PRs for features, bug fixes, or additional support (like JS builds). Please feel free to try it out and contribute! See [CONTRIBUTING.md](../../CONTRIBUTING.md) for details.
 
-> **Note:** This package currently builds on top of Drizzle's SQLite integration (using `drizzle-orm/sqlite-core` types and snapshots) until Drizzle adds native IndexedDB support. The migration system reads Drizzle's SQLite snapshots and translates them into IndexedDB object stores and indexes.
+> **Note:** This package currently builds on top of Drizzle's SQLite integration (using `drizzle-orm/sqlite-core` types) until Drizzle adds native IndexedDB support. The migration system uses function-based migrations generated from Drizzle's SQLite migrations to create IndexedDB object stores and indexes.
 
 ## Installation
 
@@ -19,8 +19,7 @@ npm install @firtoz/drizzle-indexeddb @firtoz/drizzle-utils drizzle-orm @tanstac
 - 🔍 **Query optimization** - Leverage IndexedDB indexes for fast queries
 - 📦 **Soft deletes** - Built-in support for `deletedAt` column
 - ⚛️ **React hooks** - Provider and hooks for easy React integration
-- 🔄 **Snapshot-based migrations** - Use Drizzle's generated snapshots to migrate IndexedDB
-- 📝 **Function-based migrations** - Write custom migration functions for complex changes
+- 📝 **Function-based migrations** - Generated migration functions from Drizzle schema changes
 
 ## Quick Start
 
@@ -40,22 +39,25 @@ export const todoTable = syncableTable("todos", {
 ### 2. Generate Migrations
 
 ```bash
-# Generate Drizzle snapshots
+# Generate Drizzle migrations
 drizzle-kit generate
+
+# Generate IndexedDB migration functions
+bun drizzle-indexeddb-generate
 ```
 
 ### 3. Migrate IndexedDB
 
 ```typescript
 // db.ts
-import { migrateIndexedDB } from "@firtoz/drizzle-indexeddb";
-import journal from "./drizzle/meta/_journal.json";
-import * as snapshots from "./drizzle/snapshots";
-
-export const db = await migrateIndexedDB("my-app", {
-  journal,
-  snapshots,
-}, true); // Enable debug logging
+import { migrateIndexedDBWithFunctions } from "@firtoz/drizzle-indexeddb";
+import migrations from "./drizzle/indexeddb-migrations";
+
+export const db = await migrateIndexedDBWithFunctions(
+  "my-app",
+  migrations,
+  true // Enable debug logging
+);
 ```
 
 ### 4. Use with React
@@ -181,19 +183,19 @@ collection.find({
 
 ## Migration Methods
 
-### Snapshot-Based Migration
+### Function-Based Migration
 
-Use Drizzle's snapshot files to automatically migrate your IndexedDB schema:
+Use generated migration functions to migrate your IndexedDB schema:
 
 ```typescript
-import { migrateIndexedDB } from "@firtoz/drizzle-indexeddb";
-import journal from "./drizzle/meta/_journal.json";
-import * as snapshots from "./drizzle/snapshots";
+import { migrateIndexedDBWithFunctions } from "@firtoz/drizzle-indexeddb";
+import migrations from "./drizzle/indexeddb-migrations";
 
-const db = await migrateIndexedDB("my-app-db", {
-  journal,
-  snapshots,
-}, true); // debug flag
+const db = await migrateIndexedDBWithFunctions(
+  "my-app-db",
+  migrations,
+  true // debug flag
+);
 
 console.log("Database migrated successfully!");
 ```
@@ -219,40 +221,49 @@ interface MigrationRecord {
 }
 ```
 
-### Function-Based Migration
+### Custom Migration Functions
 
-For complex migrations that require custom logic:
+For complex migrations that require custom logic, you can write migration functions directly:
 
 ```typescript
 import { migrateIndexedDBWithFunctions } from "@firtoz/drizzle-indexeddb";
 
 const migrations = [
   // Migration 0: Initial schema
-  async (db: IDBDatabase, transaction: IDBTransaction) => {
-    const store = db.createObjectStore("todos", { keyPath: "id" });
-    store.createIndex("title", "title", { unique: false });
+  {
+    tag: "0000_initial",
+    migrate: async (db: IDBDatabase, transaction: IDBTransaction) => {
+      const store = db.createObjectStore("todos", { keyPath: "id" });
+      store.createIndex("title", "title", { unique: false });
+    },
   },
   
   // Migration 1: Add completed index
-  async (db: IDBDatabase, transaction: IDBTransaction) => {
-    const store = transaction.objectStore("todos");
-    store.createIndex("completed", "completed", { unique: false });
+  {
+    tag: "0001_add_completed",
+    migrate: async (db: IDBDatabase, transaction: IDBTransaction) => {
+      const store = transaction.objectStore("todos");
+      store.createIndex("completed", "completed", { unique: false });
+    },
   },
   
   // Migration 2: Transform data
-  async (db: IDBDatabase, transaction: IDBTransaction) => {
-    const store = transaction.objectStore("todos");
-    const todos = await new Promise<any[]>((resolve, reject) => {
-      const req = store.getAll();
-      req.onsuccess = () => resolve(req.result);
-      req.onerror = () => reject(req.error);
-    });
-    
-    // Transform data
-    for (const todo of todos) {
-      todo.priority = todo.priority || "medium";
-      store.put(todo);
-    }
+  {
+    tag: "0002_add_priority",
+    migrate: async (db: IDBDatabase, transaction: IDBTransaction) => {
+      const store = transaction.objectStore("todos");
+      const todos = await new Promise<any[]>((resolve, reject) => {
+        const req = store.getAll();
+        req.onsuccess = () => resolve(req.result);
+        req.onerror = () => reject(req.error);
+      });
+      
+      // Transform data
+      for (const todo of todos) {
+        todo.priority = todo.priority || "medium";
+        store.put(todo);
+      }
+    },
   },
 ];
 
@@ -338,6 +349,56 @@ Useful for:
 - Clearing user data on logout
 - Testing scenarios
 
+### generateIndexedDBMigrations
+
+Generate IndexedDB migration files from Drizzle snapshots programmatically:
+
+```typescript
+import { generateIndexedDBMigrations } from "@firtoz/drizzle-indexeddb";
+
+generateIndexedDBMigrations({
+  drizzleDir: "./drizzle",           // Path to Drizzle directory (default: ./drizzle)
+  outputDir: "./drizzle/indexeddb-migrations",  // Output directory (default: ./drizzle/indexeddb-migrations)
+});
+```
+
+## CLI
+
+The package includes a CLI tool to generate IndexedDB migrations from Drizzle schema snapshots.
+
+### Usage
+
+```bash
+# Generate migrations (run after drizzle-kit generate)
+bun drizzle-indexeddb-generate
+
+# With custom paths
+bun drizzle-indexeddb-generate --drizzle-dir ./db/drizzle
+bun drizzle-indexeddb-generate --output-dir ./src/migrations
+
+# Show help
+bun drizzle-indexeddb-generate --help
+```
+
+### Options
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `--drizzle-dir <path>` | Path to Drizzle directory | `./drizzle` |
+| `--output-dir <path>` | Path to output directory | `./drizzle/indexeddb-migrations` |
+
+### npm scripts
+
+Add to your `package.json`:
+
+```json
+{
+  "scripts": {
+    "db:generate": "bun drizzle-kit generate && bun drizzle-indexeddb-generate"
+  }
+}
+```
+
 ## Advanced Usage
 
 ### Custom Sync Configuration
@@ -359,13 +420,13 @@ const collection = createCollection(
 
 ```typescript
 try {
-  const db = await migrateIndexedDB("my-app", config, true);
+  const db = await migrateIndexedDBWithFunctions("my-app", migrations, true);
 } catch (error) {
   console.error("Migration failed:", error);
   
   // Option 1: Delete and start fresh
   await deleteIndexedDB("my-app");
-  const db = await migrateIndexedDB("my-app", config, true);
+  const db = await migrateIndexedDBWithFunctions("my-app", migrations, true);
   
   // Option 2: Handle specific errors
   if (error.message.includes("Primary key structure changed")) {
@@ -378,10 +439,10 @@ try {
 
 ```typescript
 // Enable debug mode to see performance metrics
-const db = await migrateIndexedDB("my-app", config, true);
+const db = await migrateIndexedDBWithFunctions("my-app", migrations, true);
 
 // Output shows:
-// [PERF] IndexedDB snapshot migrator start for my-app
+// [PERF] IndexedDB function migrator start for my-app
 // [PERF] Latest applied migration index: 5 (checked 5 migrations)
 // [PERF] Found 2 pending migrations to apply: ["add_priority", "add_category"]
 // [PERF] Upgrade started: v5 → v7
@@ -432,9 +493,9 @@ const todoTable = syncableTable("todos", {
 
 ### Renaming a Column
 
-Drizzle snapshots don't track renames directly, but you can:
+Drizzle migrations don't track renames directly, but you can:
 
-1. Use function-based migrations to handle data transformation
+1. Modify the generated migration function to handle data transformation
 2. Or: Add new column, copy data, delete old column (3 separate migrations)
 
 ### Deleting a Table
@@ -455,8 +516,8 @@ This happens when you change the primary key of a table. IndexedDB doesn't suppo
 
 ### Migrations Not Applying
 
-- Check that journal and snapshots are correctly imported
-- Verify the snapshot files exist in `drizzle/snapshots/`
+- Check that migrations are correctly imported from `drizzle/indexeddb-migrations/`
+- Verify the migration files exist - run `bun drizzle-indexeddb-generate` to regenerate
 - Enable debug mode to see what's happening
 - Check browser DevTools → Application → IndexedDB
 

package/package.json

@@ -1,17 +1,25 @@
 {
 	"name": "@firtoz/drizzle-indexeddb",
-	"version": "0.2.0",
-	"description": "IndexedDB migrations powered by Drizzle ORM snapshots",
+	"version": "0.3.0",
+	"description": "IndexedDB migrations powered by Drizzle ORM",
 	"main": "./src/index.ts",
 	"module": "./src/index.ts",
 	"types": "./src/index.ts",
 	"type": "module",
+	"bin": {
+		"drizzle-indexeddb-generate": "./src/bin/generate-migrations.ts"
+	},
 	"exports": {
 		".": {
 			"types": "./src/index.ts",
 			"import": "./src/index.ts",
 			"require": "./src/index.ts"
 		},
+		"./generate": {
+			"types": "./src/bin/generate-migrations.ts",
+			"import": "./src/bin/generate-migrations.ts",
+			"require": "./src/bin/generate-migrations.ts"
+		},
 		"./*": {
 			"types": "./src/*.ts",
 			"import": "./src/*.ts",
@@ -57,15 +65,15 @@
 	},
 	"dependencies": {
 		"@firtoz/drizzle-utils": "^0.2.0",
-		"@tanstack/db": "^0.5.0",
+		"@tanstack/db": "^0.5.10",
 		"drizzle-orm": "^0.44.7",
 		"drizzle-valibot": "^0.4.2",
-		"valibot": "^1.0.0"
+		"valibot": "^1.2.0"
 	},
 	"peerDependencies": {
 		"react": "^19.2.0"
 	},
 	"devDependencies": {
-		"@types/react": "^19.2.5"
+		"@types/react": "^19.2.7"
 	}
 }

package/src/bin/generate-migrations.ts

@@ -0,0 +1,288 @@
+#!/usr/bin/env node
+/**
+ * CLI tool to generate IndexedDB migrations from Drizzle schema snapshots
+ * Run after `drizzle-kit generate` to create executable migration files
+ *
+ * Usage:
+ *   bun drizzle-indexeddb-generate
+ */
+
+import { readFileSync, writeFileSync, mkdirSync, existsSync } from "node:fs";
+import { join, resolve } from "node:path";
+import type {
+	JournalEntry,
+	Journal,
+	Snapshot,
+	TableDefinition,
+	ColumnDefinition,
+	IndexDefinition,
+} from "@firtoz/drizzle-utils";
+import type { Migration, MigrationOperation } from "../function-migrator";
+
+interface GenerateOptions {
+	drizzleDir?: string;
+	outputDir?: string;
+}
+
+function generateMigration(
+	_entry: JournalEntry,
+	snapshot: Snapshot,
+	prevSnapshot: Snapshot | null,
+): Migration {
+	const operations: MigrationOperation[] = [];
+
+	const currentTables: Record<string, TableDefinition> = snapshot.tables || {};
+	const previousTables: Record<string, TableDefinition> =
+		prevSnapshot?.tables || {};
+
+	// Find new tables
+	for (const [tableName, tableDef] of Object.entries(currentTables)) {
+		if (!previousTables[tableName]) {
+			// Find primary key
+			const pkColumn = Object.values(
+				tableDef.columns as Record<string, ColumnDefinition>,
+			).find((col) => col.primaryKey);
+
+			const indexes: Array<{
+				name: string;
+				keyPath: string | string[];
+				unique?: boolean;
+			}> = [];
+
+			// Collect indexes
+			for (const [indexName, indexDef] of Object.entries(
+				tableDef.indexes || {},
+			)) {
+				indexes.push({
+					name: indexName,
+					keyPath:
+						indexDef.columns.length === 1
+							? indexDef.columns[0]
+							: indexDef.columns,
+					unique: indexDef.isUnique,
+				});
+			}
+
+			operations.push({
+				type: "createTable",
+				name: tableName,
+				keyPath: pkColumn?.name,
+				autoIncrement: pkColumn?.autoincrement ?? false,
+				indexes: indexes.length > 0 ? indexes : undefined,
+			});
+		} else {
+			// Table exists, check for index changes
+			const prevTableDef: TableDefinition = previousTables[tableName];
+			const newIndexes: Record<string, IndexDefinition> =
+				tableDef.indexes || {};
+			const oldIndexes: Record<string, IndexDefinition> =
+				prevTableDef.indexes || {};
+
+			// Add new indexes
+			for (const [indexName, indexDef] of Object.entries(newIndexes)) {
+				if (!oldIndexes[indexName]) {
+					operations.push({
+						type: "createIndex",
+						tableName,
+						indexName,
+						keyPath:
+							indexDef.columns.length === 1
+								? indexDef.columns[0]
+								: indexDef.columns,
+						unique: indexDef.isUnique,
+					});
+				}
+			}
+
+			// Delete removed indexes
+			for (const indexName of Object.keys(oldIndexes)) {
+				if (!newIndexes[indexName]) {
+					operations.push({
+						type: "deleteIndex",
+						tableName,
+						indexName,
+					});
+				}
+			}
+		}
+	}
+
+	// Find deleted tables
+	for (const tableName of Object.keys(previousTables)) {
+		if (!currentTables[tableName]) {
+			operations.push({
+				type: "deleteTable",
+				name: tableName,
+			});
+		}
+	}
+
+	return operations;
+}
+
+function migrationToCode(
+	migration: Migration,
+	idx: number,
+	tag: string,
+): string {
+	const migrationName = tag.replace(/^\d+_/, "").replace(/_/g, " ");
+	const lines: string[] = [];
+
+	lines.push(
+		`import type { Migration } from "@firtoz/drizzle-indexeddb";`,
+		``,
+		`/**`,
+		` * Migration: ${migrationName}`,
+		` * Generated from: ${tag}`,
+		` */`,
+		`export const migrate_${idx.toString().padStart(4, "0")}: Migration = ${JSON.stringify(migration, null, "\t")};`,
+	);
+
+	return lines.join("\n");
+}
+
+export function generateIndexedDBMigrations(
+	options: GenerateOptions = {},
+): void {
+	const cwd = process.cwd();
+	const drizzleDir = resolve(cwd, options.drizzleDir || "./drizzle");
+	const metaDir = join(drizzleDir, "meta");
+	const journalPath = join(metaDir, "_journal.json");
+	const outputDir = resolve(
+		cwd,
+		options.outputDir || join(drizzleDir, "indexeddb-migrations"),
+	);
+
+	const startTime = performance.now();
+	console.log(`[drizzle-indexeddb] Starting migration generation...`);
+
+	// Read the journal
+	if (!existsSync(journalPath)) {
+		console.error(
+			`[drizzle-indexeddb] Error: Journal not found at ${journalPath}`,
+		);
+		console.error(
+			`[drizzle-indexeddb] Make sure to run 'drizzle-kit generate' first`,
+		);
+		process.exit(1);
+	}
+
+	const journalContent = readFileSync(journalPath, "utf-8");
+	const journal: Journal = JSON.parse(journalContent);
+
+	console.log(`[drizzle-indexeddb] Found ${journal.entries.length} migrations`);
+
+	// Create output directory
+	if (!existsSync(outputDir)) {
+		mkdirSync(outputDir, { recursive: true });
+		console.log(`[drizzle-indexeddb] Created output directory: ${outputDir}`);
+	}
+
+	const migrationImports: string[] = [];
+	const migrationNames: string[] = [];
+
+	// Load all snapshots and generate migrations
+	const snapshots: Snapshot[] = [];
+
+	for (const entry of journal.entries) {
+		const fileName = `${entry.idx.toString().padStart(4, "0")}_snapshot.json`;
+		const snapshotPath = join(metaDir, fileName);
+
+		// Load snapshot
+		if (!existsSync(snapshotPath)) {
+			console.error(
+				`[drizzle-indexeddb] Error: Snapshot not found at ${snapshotPath}`,
+			);
+			process.exit(1);
+		}
+
+		const snapshotContent = readFileSync(snapshotPath, "utf-8");
+		const snapshot: Snapshot = JSON.parse(snapshotContent);
+		snapshots.push(snapshot);
+
+		// Generate migration
+		const prevSnapshot = entry.idx > 0 ? snapshots[entry.idx - 1] : null;
+		const migration = generateMigration(entry, snapshot, prevSnapshot);
+		const migrationCode = migrationToCode(migration, entry.idx, entry.tag);
+
+		const migrationFileName = `${entry.tag}.ts`;
+		const migrationPath = join(outputDir, migrationFileName);
+		writeFileSync(migrationPath, migrationCode, "utf-8");
+
+		// Add to index imports
+		const migrationName = `migrate_${entry.idx.toString().padStart(4, "0")}`;
+		migrationImports.push(`import { ${migrationName} } from './${entry.tag}';`);
+		migrationNames.push(migrationName);
+	}
+
+	// Generate index.ts for migrations
+	const indexContent = `import type { Migration } from "@firtoz/drizzle-indexeddb";
+
+${migrationImports.join("\n")}
+
+export const migrations: Migration[] = [
+\t${migrationNames.join(",\n\t")}
+];
+
+export default migrations;
+`;
+
+	writeFileSync(join(outputDir, "index.ts"), indexContent, "utf-8");
+	console.log(`[drizzle-indexeddb] ✓ Generated ${join(outputDir, "index.ts")}`);
+
+	const endTime = performance.now();
+	const totalTime = endTime - startTime;
+
+	console.log(`[drizzle-indexeddb] Migrations: ${migrationNames.join(", ")}`);
+	console.log(
+		`[drizzle-indexeddb] ✓ Complete! Generated ${journal.entries.length} migrations in ${totalTime.toFixed(2)}ms`,
+	);
+}
+
+// CLI entry point
+function main(): void {
+	const args = process.argv.slice(2);
+	const command = args[0];
+
+	if (command === "generate" || command === undefined) {
+		// Parse options
+		const options: GenerateOptions = {};
+
+		for (let i = 1; i < args.length; i++) {
+			const arg = args[i];
+			if (arg === "--drizzle-dir" && args[i + 1]) {
+				options.drizzleDir = args[++i];
+			} else if (arg === "--output-dir" && args[i + 1]) {
+				options.outputDir = args[++i];
+			}
+		}
+
+		generateIndexedDBMigrations(options);
+	} else if (command === "--help" || command === "-h") {
+		console.log(`
+drizzle-indexeddb-generate - Generate IndexedDB migrations from Drizzle schema
+
+Usage:
+  bun drizzle-indexeddb-generate [options]
+
+Options:
+  --drizzle-dir <path>  Path to Drizzle directory (default: ./drizzle)
+  --output-dir <path>   Path to output directory (default: ./drizzle/indexeddb-migrations)
+  -h, --help            Show this help message
+
+Examples:
+  bun drizzle-indexeddb-generate
+  bun drizzle-indexeddb-generate --drizzle-dir ./db/drizzle
+  bun drizzle-indexeddb-generate --output-dir ./src/migrations
+`);
+	} else {
+		console.error(`Unknown command: ${command}`);
+		console.error(`Run 'bun drizzle-indexeddb-generate --help' for usage`);
+		process.exit(1);
+	}
+}
+
+// Only run CLI when executed directly (not when imported)
+if (import.meta.main) {
+	main();
+}