remult-sqlite-s3 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,145 @@
+# remult-sqlite-s3
+
+A [Remult](https://remult.dev) data provider that wraps [better-sqlite3](https://github.com/WiseLibs/better-sqlite3) with automatic S3 synchronization.
+
+## Installation
+
+```bash
+npm install remult-sqlite-s3 better-sqlite3
+```
+
+## Usage
+
+```typescript
+import { remultApi } from 'remult/remult-express'
+import { SqlDatabase } from 'remult'
+import { createS3DataProvider } from 'remult-sqlite-s3'
+import { Task } from './entities/Task'
+
+export const api = remultApi({
+  dataProvider: createS3DataProvider({
+    file: './mydb.sqlite',
+    s3: {
+      bucket: 'my-bucket',
+      databaseName: 'my-database',
+    },
+  }),
+  entities: [Task],
+})
+```
+
+The `databaseName` is required and must be unique per database to prevent data conflicts in S3.
+
+## S3 Configuration
+
+### AWS S3
+
+```typescript
+s3: {
+  bucket: 'my-bucket',
+  databaseName: 'my-database',
+  region: 'us-east-1',  // optional, defaults to us-east-1
+  credentials: {        // optional, uses AWS SDK default chain if not provided
+    accessKeyId: '...',
+    secretAccessKey: '...',
+  },
+}
+```
+
+### Cloudflare R2
+
+```typescript
+s3: {
+  bucket: 'my-bucket',
+  databaseName: 'my-database',
+  endpoint: 'https://<account-id>.r2.cloudflarestorage.com',
+  credentials: {
+    accessKeyId: process.env.R2_ACCESS_KEY_ID!,
+    secretAccessKey: process.env.R2_SECRET_ACCESS_KEY!,
+  },
+}
+```
+
+### MinIO
+
+```typescript
+s3: {
+  bucket: 'my-bucket',
+  databaseName: 'my-database',
+  endpoint: 'http://localhost:9000',
+  forcePathStyle: true,
+  credentials: {
+    accessKeyId: 'minioadmin',
+    secretAccessKey: 'minioadmin',
+  },
+}
+```
+
+## Options
+
+| Option | Description | Required | Default |
+|--------|-------------|----------|---------|
+| `file` | Path to SQLite database file | Yes | - |
+| `s3.bucket` | S3 bucket name | Yes | - |
+| `s3.databaseName` | Unique name for this database | Yes | - |
+| `s3.keyPrefix` | Prefix for S3 keys (for organization) | No | - |
+| `s3.region` | AWS region | No | `us-east-1` |
+| `s3.endpoint` | Custom endpoint for S3-compatible storage | No | - |
+| `s3.forcePathStyle` | Use path-style URLs (required for MinIO) | No | `false` |
+| `s3.credentials` | AWS credentials | No | SDK default |
+| `sync.mode` | `async` (fast) or `sync` (durable) | No | `async` |
+| `sync.snapshotInterval` | Interval between snapshots (ms) | No | `300000` |
+| `sync.lockTtl` | Lock TTL (ms) | No | `60000` |
+| `sync.forceLock` | Force acquire lock (dev only) | No | `false` |
+| `verbose` | Enable verbose logging | No | `false` |
+
+## How It Works
+
+1. **On startup**: Acquires a distributed lock, downloads the latest snapshot from S3 if local database is missing
+2. **On writes**: Syncs changes to S3 (immediately in `sync` mode, periodically in `async` mode)
+3. **On shutdown**: Takes a final snapshot and releases the lock
+
+### S3 Structure
+
+```
+[keyPrefix/]<databaseName>/
+├── lock.json
+└── generations/<id>/
+    ├── snapshot.db
+    └── snapshot.meta.json
+```
+
+## Framework Examples
+
+### SvelteKit
+
+See the [examples/sveltekit](./examples/sveltekit) folder for a complete working example.
+
+```typescript
+// src/server/api.ts
+import { remultApi } from 'remult/remult-sveltekit'
+import { createS3DataProvider } from 'remult-sqlite-s3'
+import { building } from '$app/environment'
+import { env } from '$env/dynamic/private'
+
+export const api = remultApi({
+  dataProvider: building ? undefined : createS3DataProvider({
+    file: './mydb.sqlite',
+    s3: {
+      bucket: env.S3_BUCKET!,
+      databaseName: env.DATABASE_NAME!,
+    },
+  }),
+  entities: [...],
+})
+```
+
+## Limitations
+
+- **Single writer**: Only one instance can write at a time (by design)
+- **Cold starts**: Initial recovery from S3 adds startup latency
+- **Best for**: Low-to-medium write loads in serverless environments
+
+## License
+
+MIT