spindb 0.13.2 → 0.13.4

package/README.md

@@ -429,17 +429,24 @@ spindb backup mydb --output ./backups/      # Custom directory
 spindb backup mydb --database my_app        # Backup specific database
 ```
 
-Backup formats:
+Backup formats (vary by engine):
 
 ```bash
-spindb backup mydb --format sql     # Plain SQL (.sql)
-spindb backup mydb --format dump    # Compressed (.dump for PG, .sql.gz for MySQL)
+spindb backup mydb --format sql     # Plain SQL (.sql) or text commands (.redis)
+spindb backup mydb --format dump    # Binary format (.dump for PG, .sql.gz for MySQL, .rdb for Redis)
 
 # Shorthand
 spindb backup mydb --sql
 spindb backup mydb --dump
 ```
 
+Format by engine:
+- PostgreSQL: `.sql` (plain SQL) / `.dump` (pg_dump custom)
+- MySQL: `.sql` (plain SQL) / `.sql.gz` (compressed SQL)
+- SQLite: `.sql` (plain SQL) / `.sqlite` (binary copy)
+- MongoDB: `.bson` (BSON dump) / `.archive` (compressed archive)
+- Redis: `.redis` (text commands) / `.rdb` (RDB snapshot)
+
 <details>
 <summary>All options</summary>
 
@@ -455,6 +462,27 @@ spindb backup mydb --dump
 
 </details>
 
+#### `backups` - List backup files
+
+```bash
+spindb backups                       # List backups in current directory
+spindb backups ./data                # List backups in specific directory
+spindb backups --all                 # Include ~/.spindb/backups
+spindb backups --limit 50            # Show more results
+spindb backups --json                # JSON output
+```
+
+<details>
+<summary>All options</summary>
+
+| Option | Description |
+|--------|-------------|
+| `--all`, `-a` | Include backups from `~/.spindb/backups` |
+| `--limit`, `-n` | Limit number of results (default: 20) |
+| `--json`, `-j` | Output as JSON |
+
+</details>
+
 #### `restore` - Restore from backup
 
 ```bash
@@ -488,6 +516,78 @@ spindb info mydb
 
 </details>
 
+#### Backup & Restore Format Reference
+
+Each engine has specific backup formats and restore behaviors:
+
+<details>
+<summary>PostgreSQL</summary>
+
+| Format | Extension | Tool | Notes |
+|--------|-----------|------|-------|
+| SQL | `.sql` | pg_dump | Plain text SQL, human-readable |
+| Custom | `.dump` | pg_dump -Fc | Compressed, supports parallel restore |
+
+**Restore behavior:** Creates new database or replaces existing. Uses `pg_restore` for `.dump`, `psql` for `.sql`.
+
+</details>
+
+<details>
+<summary>MySQL</summary>
+
+| Format | Extension | Tool | Notes |
+|--------|-----------|------|-------|
+| SQL | `.sql` | mysqldump | Plain text SQL |
+| Compressed | `.sql.gz` | mysqldump + gzip | Gzip compressed SQL |
+
+**Restore behavior:** Creates new database or replaces existing. Pipes to `mysql` client.
+
+</details>
+
+<details>
+<summary>SQLite</summary>
+
+| Format | Extension | Tool | Notes |
+|--------|-----------|------|-------|
+| SQL | `.sql` | .dump | Plain text SQL |
+| Binary | `.sqlite` | File copy | Exact copy of database file |
+
+**Restore behavior:** Creates new file or replaces existing.
+
+</details>
+
+<details>
+<summary>MongoDB</summary>
+
+| Format | Extension | Tool | Notes |
+|--------|-----------|------|-------|
+| BSON | `.bson` | mongodump | Binary JSON per collection |
+| Archive | `.archive` | mongodump --archive | Single compressed file |
+
+**Restore behavior:** Creates new database or replaces existing. Uses `mongorestore`.
+
+</details>
+
+<details>
+<summary>Redis</summary>
+
+| Format | Extension | Tool | Notes |
+|--------|-----------|------|-------|
+| RDB | `.rdb` | BGSAVE | Binary snapshot, requires restart |
+| Text | `.redis` | Custom | Human-readable Redis commands |
+
+**Text format detection:** Files are detected as Redis text commands if they contain valid Redis commands (SET, HSET, DEL, etc.), regardless of file extension. This allows restoring files like `users.txt` or `data` without renaming.
+
+**Restore behavior:**
+- **RDB (`.rdb`):** Requires stopping Redis, copies file to data directory, restart loads data
+- **Text (`.redis`):** Pipes commands to running Redis instance. Prompts for:
+  - **Replace all:** Runs `FLUSHDB` first (clean slate)
+  - **Merge:** Adds/updates keys, keeps existing keys not in backup
+
+**Note:** Redis uses numbered databases (0-15) that always exist. "Create new database" is not applicable.
+
+</details>
+
 ### Container Management
 
 #### `list` - List all containers
@@ -699,7 +799,7 @@ Native processes mean instant startup and no virtualization overhead.
 └── mydb.sqlite                             # Created with: spindb create mydb -e sqlite
 ```
 
-### How Data Persists
+### Data Persistence
 
 SpinDB runs databases as **native processes** on your machine. When you start a container:
 
@@ -717,6 +817,29 @@ When you stop a container:
 
 **Your data is never deleted unless you explicitly delete the container.**
 
+#### Persistence by Engine
+
+Each database engine has its own persistence mechanism:
+
+| Engine | Mechanism | Durability |
+|--------|-----------|------------|
+| PostgreSQL | Write-Ahead Logging (WAL) | Every commit is immediately durable |
+| MySQL | InnoDB transaction logs | Every commit is immediately durable |
+| SQLite | File-based transactions | Every commit is immediately durable |
+| MongoDB | WiredTiger with journaling | Writes journaled before acknowledged |
+| Redis | RDB snapshots | Periodic snapshots (see below) |
+
+**PostgreSQL, MySQL, MongoDB:** These engines use transaction logs or journaling. Every committed write is guaranteed to survive a crash or unexpected shutdown.
+
+**SQLite:** As a file-based database, SQLite writes directly to disk on each commit. No server process means no risk of losing in-flight data.
+
+**Redis:** SpinDB configures Redis with RDB (Redis Database) snapshots:
+- Save after 900 seconds if at least 1 key changed
+- Save after 300 seconds if at least 10 keys changed
+- Save after 60 seconds if at least 10,000 keys changed
+
+This means Redis may lose up to ~60 seconds of writes on an unexpected crash. For local development, this trade-off (speed over strict durability) is typically acceptable. If you need stronger guarantees, use `spindb backup` before stopping work.
+
 ### Binary Sources
 
 **PostgreSQL:** Server binaries are downloaded automatically:
@@ -738,14 +861,15 @@ This isn't a preference—it's a practical reality of what's available.
 
 This makes multi-version support trivial: need PostgreSQL 14 for a legacy project and 18 for a new one? SpinDB downloads both, and they run side-by-side without conflicts.
 
-**No equivalent exists for MySQL or MongoDB.** Neither database has a comparable embedded binary project:
+**No equivalent exists for MySQL, MongoDB, or Redis.** None of these databases have a comparable embedded binary project:
 
 - **MySQL:** Oracle distributes MySQL as large installers with system dependencies, not embeddable binaries. There's no "zonky.io for MySQL."
-- **MongoDB:** Server binaries are several hundred MB with complex licensing around redistribution. MongoDB Inc. doesn't provide an embedded distribution.
+- **MongoDB:** Server binaries are several hundred MB and aren't designed for portable distribution.
+- **Redis:** While Redis is small (~6-12 MB), there's no official portable distribution. Community Windows ports exist, but macOS/Linux rely on system packages.
 
 For these databases, system packages (Homebrew, apt, choco) are the most reliable option. They handle dependencies, platform quirks, and security updates. SpinDB simply orchestrates what's already installed.
 
-**Does this limit multi-version support?** Yes, for MySQL/MongoDB you get whatever version your package manager provides. In practice, this is rarely a problem—developers seldom need multiple MySQL versions simultaneously. If zonky.io-style distributions emerged for other databases, SpinDB could adopt them.
+**Does this limit multi-version support?** Yes, for MySQL/MongoDB/Redis you get whatever version your package manager provides. In practice, this is rarely a problem—developers seldom need multiple versions of these databases simultaneously. If zonky.io-style distributions emerged for other databases, SpinDB could adopt them.
 
 ---
 
@@ -776,6 +900,20 @@ See [TODO.md](TODO.md) for the full roadmap.
 - Scheduled backups
 - Import from Docker
 
+### Possible Future Engines
+
+These engines are under consideration but not yet on the roadmap. Community interest and feasibility will determine priority:
+
+| Engine | Type | Notes |
+|--------|------|-------|
+| **DuckDB** | Embedded analytical | File-based like SQLite, popular for data/analytics work |
+| **libSQL** | Embedded relational | SQLite fork by Turso with replication and edge support |
+| **Valkey** | Key-value store | Redis fork (post-license change), growing adoption |
+| **Meilisearch** | Search engine | Developer-friendly search, good binary distribution |
+| **Elasticsearch/OpenSearch** | Search engine | Full-text search, common in web applications |
+| **Neo4j** | Graph database | Most popular graph database |
+| **InfluxDB** | Time-series | IoT, metrics, and monitoring use cases |
+
 ---
 
 ## Troubleshooting
@@ -838,6 +976,14 @@ SpinDB wouldn't be possible without:
 
 ---
 
+## Related Work
+
+We're actively contributing to the broader embedded database ecosystem:
+
+- **[hostdb](https://github.com/robertjbass/hostdb)** - A companion project providing downloadable database binaries (Redis, MySQL/MariaDB, etc.) as GitHub releases. This will enable SpinDB to offer multi-version support for additional engines beyond PostgreSQL.
+
+---
+
 ## License
 
 [PolyForm Noncommercial 1.0.0](LICENSE)

package/cli/commands/backup.ts

@@ -29,11 +29,39 @@ function generateDefaultFilename(
 }
 
 function getExtension(format: 'sql' | 'dump', engine: string): string {
+  // Handle 'sql' format (human-readable option)
   if (format === 'sql') {
-    return '.sql'
+    // MongoDB uses BSON directory format for 'sql' choice
+    return engine === 'mongodb' ? '' : '.sql'
+  }
+
+  // Handle 'dump' format (binary/compressed option)
+  switch (engine) {
+    case 'mysql':
+      return '.sql.gz'
+    case 'sqlite':
+      return '.sqlite'
+    case 'mongodb':
+      return '.archive'
+    case 'redis':
+      return '.rdb'
+    case 'postgresql':
+    default:
+      return '.dump'
   }
-  // MySQL dump is gzipped SQL, PostgreSQL dump is custom format
-  return engine === 'mysql' ? '.sql.gz' : '.dump'
+}
+
+function getFormatDescription(format: 'sql' | 'dump', engine: string): string {
+  if (engine === 'redis') {
+    return 'RDB snapshot'
+  }
+  if (engine === 'mongodb') {
+    return format === 'sql' ? 'BSON directory' : 'archive'
+  }
+  if (engine === 'sqlite') {
+    return format === 'sql' ? 'SQL' : 'binary'
+  }
+  return format === 'sql' ? 'SQL' : 'dump'
 }
 
 export const backupCommand = new Command('backup')
@@ -195,8 +223,9 @@ export const backupCommand = new Command('backup')
         const outputDir = options.output || process.cwd()
         const outputPath = join(outputDir, `${filename}${extension}`)
 
+        const formatDesc = getFormatDescription(format, engineName)
         const backupSpinner = createSpinner(
-          `Creating ${format === 'sql' ? 'SQL' : 'dump'} backup of "${databaseName}"...`,
+          `Creating ${formatDesc} backup of "${databaseName}"...`,
         )
         backupSpinner.start()
 
@@ -222,11 +251,8 @@ export const backupCommand = new Command('backup')
           console.log()
           console.log(uiSuccess('Backup complete'))
           console.log()
-          console.log(chalk.gray('  File:'), chalk.cyan(result.path))
-          console.log(
-            chalk.gray('  Size:'),
-            chalk.white(formatBytes(result.size)),
-          )
+          console.log(chalk.gray('  Saved to:'), chalk.cyan(result.path))
+          console.log(chalk.gray('  Size:'), chalk.white(formatBytes(result.size)))
           console.log(chalk.gray('  Format:'), chalk.white(result.format))
           console.log()
         }

package/cli/commands/backups.ts

@@ -0,0 +1,259 @@
+/**
+ * List backups command
+ *
+ * Scans the current directory (or specified directory) for backup files
+ * and displays them with metadata.
+ */
+
+import { Command } from 'commander'
+import { readdirSync, statSync } from 'fs'
+import { join, extname } from 'path'
+import chalk from 'chalk'
+import { formatBytes } from '../ui/theme'
+
+type BackupInfo = {
+  filename: string
+  path: string
+  size: number
+  modified: Date
+  engine: string | null
+  format: string
+}
+
+/**
+ * Detect engine and format from file extension
+ */
+function detectBackupType(filename: string): { engine: string | null; format: string } {
+  const ext = extname(filename).toLowerCase()
+
+  // Check for double extensions like .sql.gz
+  if (filename.endsWith('.sql.gz')) {
+    return { engine: 'mysql', format: 'Compressed SQL' }
+  }
+
+  switch (ext) {
+    case '.sql':
+      // Could be PostgreSQL, MySQL, or SQLite
+      return { engine: null, format: 'SQL dump' }
+    case '.dump':
+      return { engine: 'postgresql', format: 'pg_dump custom' }
+    case '.sqlite':
+    case '.db':
+    case '.sqlite3':
+      return { engine: 'sqlite', format: 'Binary copy' }
+    case '.archive':
+      return { engine: 'mongodb', format: 'BSON archive' }
+    case '.rdb':
+      return { engine: 'redis', format: 'RDB snapshot' }
+    case '.redis':
+      return { engine: 'redis', format: 'Text commands' }
+    case '.bson':
+      return { engine: 'mongodb', format: 'BSON' }
+    default:
+      return { engine: null, format: 'Unknown' }
+  }
+}
+
+/**
+ * Check if a file looks like a backup file
+ */
+function isBackupFile(filename: string): boolean {
+  const backupExtensions = [
+    '.sql',
+    '.dump',
+    '.sqlite',
+    '.sqlite3',
+    '.db',
+    '.archive',
+    '.rdb',
+    '.redis',
+    '.bson',
+  ]
+
+  // Check for .sql.gz
+  if (filename.endsWith('.sql.gz')) return true
+
+  const ext = extname(filename).toLowerCase()
+  return backupExtensions.includes(ext)
+}
+
+/**
+ * Scan directory for backup files
+ */
+function findBackups(directory: string): BackupInfo[] {
+  const backups: BackupInfo[] = []
+
+  try {
+    const files = readdirSync(directory)
+
+    for (const file of files) {
+      if (!isBackupFile(file)) continue
+
+      const filePath = join(directory, file)
+      try {
+        const stats = statSync(filePath)
+        if (!stats.isFile()) continue
+
+        const { engine, format } = detectBackupType(file)
+
+        backups.push({
+          filename: file,
+          path: filePath,
+          size: stats.size,
+          modified: stats.mtime,
+          engine,
+          format,
+        })
+      } catch {
+        // Skip files we can't stat
+      }
+    }
+  } catch {
+    // Directory doesn't exist or can't be read
+  }
+
+  // Sort by modified date, newest first
+  backups.sort((a, b) => b.modified.getTime() - a.modified.getTime())
+
+  return backups
+}
+
+/**
+ * Format a relative time string
+ */
+function formatRelativeTime(date: Date): string {
+  const now = new Date()
+  const diffMs = now.getTime() - date.getTime()
+  const diffMins = Math.floor(diffMs / 60000)
+  const diffHours = Math.floor(diffMs / 3600000)
+  const diffDays = Math.floor(diffMs / 86400000)
+
+  if (diffMins < 1) return 'just now'
+  if (diffMins < 60) return `${diffMins}m ago`
+  if (diffHours < 24) return `${diffHours}h ago`
+  if (diffDays < 7) return `${diffDays}d ago`
+
+  return date.toLocaleDateString()
+}
+
+/**
+ * Get engine icon
+ */
+function getEngineIcon(engine: string | null): string {
+  switch (engine) {
+    case 'postgresql':
+      return '🐘'
+    case 'mysql':
+      return '🐬'
+    case 'sqlite':
+      return '🗄️'
+    case 'mongodb':
+      return '🍃'
+    case 'redis':
+      return '🔴'
+    default:
+      return '📦'
+  }
+}
+
+export const backupsCommand = new Command('backups')
+  .description('List backup files in current directory')
+  .argument('[directory]', 'Directory to scan (defaults to current directory)')
+  .option('-a, --all', 'Include backups from ~/.spindb/backups as well')
+  .option('-n, --limit <count>', 'Limit number of results', '20')
+  .option('-j, --json', 'Output as JSON')
+  .action(
+    async (
+      directory: string | undefined,
+      options: {
+        all?: boolean
+        limit?: string
+        json?: boolean
+      },
+    ) => {
+      const searchDirs = [directory || process.cwd()]
+
+      if (options.all) {
+        const homeBackups = join(
+          process.env.HOME || '',
+          '.spindb',
+          'backups',
+        )
+        searchDirs.push(homeBackups)
+      }
+
+      const allBackups: BackupInfo[] = []
+
+      for (const dir of searchDirs) {
+        const backups = findBackups(dir)
+        allBackups.push(...backups)
+      }
+
+      // Sort all backups by date
+      allBackups.sort((a, b) => b.modified.getTime() - a.modified.getTime())
+
+      // Apply limit
+      const limit = parseInt(options.limit || '20', 10)
+      const limitedBackups = allBackups.slice(0, limit)
+
+      if (options.json) {
+        console.log(
+          JSON.stringify(
+            limitedBackups.map((b) => ({
+              filename: b.filename,
+              path: b.path,
+              size: b.size,
+              modified: b.modified.toISOString(),
+              engine: b.engine,
+              format: b.format,
+            })),
+            null,
+            2,
+          ),
+        )
+        return
+      }
+
+      if (limitedBackups.length === 0) {
+        console.log()
+        console.log(chalk.gray('  No backup files found'))
+        console.log()
+        console.log(chalk.gray('  Backup files are identified by extensions:'))
+        console.log(chalk.gray('    .sql, .dump, .sqlite, .archive, .rdb, .sql.gz'))
+        console.log()
+        return
+      }
+
+      console.log()
+      console.log(chalk.bold(`  Found ${allBackups.length} backup(s)`))
+      if (allBackups.length > limit) {
+        console.log(chalk.gray(`  (showing ${limit} most recent)`))
+      }
+      console.log()
+
+      // Calculate column widths
+      const maxFilename = Math.min(
+        50,
+        Math.max(...limitedBackups.map((b) => b.filename.length)),
+      )
+
+      for (const backup of limitedBackups) {
+        const icon = getEngineIcon(backup.engine)
+        const filename =
+          backup.filename.length > maxFilename
+            ? backup.filename.slice(0, maxFilename - 3) + '...'
+            : backup.filename.padEnd(maxFilename)
+
+        const size = formatBytes(backup.size).padStart(10)
+        const time = formatRelativeTime(backup.modified).padStart(10)
+        const format = chalk.gray(backup.format)
+
+        console.log(`  ${icon} ${chalk.cyan(filename)} ${chalk.white(size)} ${chalk.gray(time)} ${format}`)
+      }
+
+      console.log()
+      console.log(chalk.gray('  Restore with:'))
+      console.log(chalk.cyan('    spindb restore <container> <backup-file>'))
+      console.log()
+    },
+  )

package/cli/commands/create.ts

@@ -143,7 +143,6 @@ async function createSqliteContainer(
     console.log(chalk.cyan(`    ${connectionString}`))
     console.log()
 
-    // Connect if requested
     if (connect) {
       const config = await containerManager.getConfig(containerName)
       if (config) {
@@ -445,26 +444,26 @@ export const createCommand = new Command('create')
           depsSpinner.succeed('Required tools available')
         }
 
-        // For MySQL (and other non-PostgreSQL server DBs), download binaries after dep check
+        // For MySQL, MongoDB, Redis (system-installed engines), validate version and get binary path
+        // Store the binary path for version consistency
+        let binaryPath: string | undefined
         if (!isPostgreSQL) {
           const binarySpinner = createSpinner(
             `Checking ${dbEngine.displayName} ${version} binaries...`,
           )
           binarySpinner.start()
 
-          const isInstalled = await dbEngine.isBinaryInstalled(version)
-          if (isInstalled) {
-            binarySpinner.succeed(
-              `${dbEngine.displayName} ${version} binaries ready (cached)`,
-            )
-          } else {
-            binarySpinner.text = `Downloading ${dbEngine.displayName} ${version} binaries...`
-            await dbEngine.ensureBinaries(version, ({ message }) => {
+          try {
+            // ensureBinaries validates the version and returns the binary path
+            binaryPath = await dbEngine.ensureBinaries(version, ({ message }) => {
               binarySpinner.text = message
             })
             binarySpinner.succeed(
-              `${dbEngine.displayName} ${version} binaries downloaded`,
+              `${dbEngine.displayName} ${version} binaries ready`,
             )
+          } catch (error) {
+            binarySpinner.fail(`${dbEngine.displayName} ${version} not available`)
+            throw error
           }
         }
 
@@ -486,6 +485,7 @@ export const createCommand = new Command('create')
             version,
             port,
             database,
+            binaryPath,
           })
 
           tx.addRollback({