@docstack/pouchdb-adapter-googledrive 0.0.1 → 0.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +34 -19
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,13 +1,15 @@
|
|
|
1
1
|
# PouchDB Adapter for Google Drive
|
|
2
2
|
|
|
3
|
-
A PouchDB adapter that uses Google Drive as a backend storage.
|
|
3
|
+
A persistent, serverless PouchDB adapter that uses Google Drive as a backend storage. Designed for high concurrency, large datasets (via lazy loading), and offline resilience.
|
|
4
4
|
|
|
5
5
|
## Features
|
|
6
6
|
|
|
7
|
-
-
|
|
8
|
-
-
|
|
9
|
-
-
|
|
10
|
-
-
|
|
7
|
+
- **🚀 Append-Only Log**: Uses an efficient append-only log pattern for fast, conflict-free writes.
|
|
8
|
+
- **⚡ Lazy Loading**: Optimizes memory and bandwidth by loading only the **Index** into memory. Document bodies are fetched on-demand.
|
|
9
|
+
- **🛡️ Optimistic Concurrency Control**: Uses ETag-based locking on metadata to prevent race conditions and data loss during simultaneous updates.
|
|
10
|
+
- **🔄 Replication Ready**: Fully automated support for PouchDB's `sync` and `replicate` protocols (bilateral sync).
|
|
11
|
+
- **📦 Auto-Compaction**: Automatically merges logs into snapshots to keep performance high.
|
|
12
|
+
- **💾 Offline/Resilient**: Retry logic with exponential backoff handles network instability and "thundering herd" scenarios.
|
|
11
13
|
|
|
12
14
|
## Installation
|
|
13
15
|
|
|
@@ -39,29 +41,38 @@ const drive = google.drive({ version: 'v3', auth: oauth2Client });
|
|
|
39
41
|
const db = new PouchDB('my-drive-db', {
|
|
40
42
|
adapter: 'googledrive',
|
|
41
43
|
drive: drive, // valid googleapis Drive instance
|
|
42
|
-
folderId: '...', // Optional: Folder ID
|
|
44
|
+
folderId: '...', // Optional: Storage Folder ID (recommended)
|
|
43
45
|
folderName: 'my-db', // Optional: Folder name (created if not exists)
|
|
44
46
|
pollingIntervalMs: 2000, // Optional: Check for remote changes
|
|
45
|
-
compactionThreshold: 50
|
|
47
|
+
compactionThreshold: 50, // Optional: Entries before auto-compaction
|
|
48
|
+
cacheSize: 1000 // Optional: Number of document bodies to keep in LRU cache
|
|
46
49
|
});
|
|
47
|
-
```
|
|
48
50
|
|
|
49
|
-
|
|
51
|
+
// Use standard PouchDB API
|
|
52
|
+
await db.put({ _id: 'doc1', title: 'Hello Drive!' });
|
|
53
|
+
const doc = await db.get('doc1');
|
|
54
|
+
```
|
|
50
55
|
|
|
51
|
-
|
|
56
|
+
## Architecture
|
|
52
57
|
|
|
53
|
-
|
|
54
|
-
2. **`_meta.json`**: Tracks the current sequence number and active log files.
|
|
55
|
-
3. **`snapshot.json`**: Contains the full database state at a specific sequence point.
|
|
56
|
-
4. **`changes-{timestamp}.ndjson`**: New changes are appended to these newline-delimited JSON files.
|
|
58
|
+
The adapter implements a **"Remote-First"** architecture designed for scale:
|
|
57
59
|
|
|
58
|
-
###
|
|
60
|
+
### 1. Storage Structure
|
|
61
|
+
Inside your Google Drive folder, you will see:
|
|
62
|
+
- `_meta.json`: The "Lock File". Tracks the sequence number and active log pointers.
|
|
63
|
+
- `snapshot-index.json`: A lightweight map of `DocID -> { Revision, FilePointer }`. Loaded at startup.
|
|
64
|
+
- `snapshot-data.json`: Large payload files containing document bodies. **Not loaded** until requested.
|
|
65
|
+
- `changes-{seq}-{uuid}.ndjson`: Immutable append-only logs for recent updates.
|
|
59
66
|
|
|
60
|
-
|
|
61
|
-
-
|
|
62
|
-
-
|
|
67
|
+
### 2. Lazy Loading & Caching
|
|
68
|
+
- **Startup**: The client downloads only `_meta.json` and `snapshot-index.json` (~MBs even for large DBs).
|
|
69
|
+
- **Access**: `db.get(id)` checks a local **LRU Cache**. If missing, it fetches the specific file containing that document from Drive.
|
|
70
|
+
- **Sync**: `db.changes()` iterates the local index, ensuring fast replication without downloading full content.
|
|
63
71
|
|
|
64
|
-
|
|
72
|
+
### 3. Concurrency
|
|
73
|
+
- **Writes**: Every write creates a new unique `changes-*.ndjson` file.
|
|
74
|
+
- **Commit**: The adapter attempts to update `_meta.json` with an ETag check (`If-Match`).
|
|
75
|
+
- **Conflict**: If `_meta.json` was changed by another client, the write retries automatically after re-syncing the index.
|
|
65
76
|
|
|
66
77
|
## Testing
|
|
67
78
|
|
|
@@ -76,3 +87,7 @@ To run the tests, you need to provide Google Drive API credentials.
|
|
|
76
87
|
```bash
|
|
77
88
|
npm test
|
|
78
89
|
```
|
|
90
|
+
|
|
91
|
+
## License
|
|
92
|
+
|
|
93
|
+
CC-BY-SA-4.0
|