kaiwen-core-js 0.1.2 → 0.1.11

package/README.md

@@ -10,6 +10,8 @@ It is 100% interoperable with the Python `kaiwen-core` SDK, allowing seamless, c
 - **Cross-Platform Cryptography**: PBKDF2 (200k iterations) and HKDF derivations for master and app-isolated keys. Fully compatible with Python's `cryptography.hazmat` stack.
 - **AES-256-GCM**: Military-grade authenticated encryption.
 - **Logical Clock Conflict Resolution**: Decentralized, timestamp-agnostic conflict resolution across devices.
+- **Automatic Background Synchronization**: Saving, updating, and deleting records automatically schedules a background push to the server.
+- **Simplified WebSocket Channel**: One-line initialization `startSync` handles initial pull, starts socket listener, and updates UI on remote events.
 - **Chunked Asset Engine**: Securely encrypt and decrypt massive media files (videos, photos) in 1MB chunks without running out of RAM.
 - **Dependency Inversion**: Easily inject your own Storage Adapters (e.g., SQLite, AsyncStorage, IndexedDB).
 - **TypeScript First**: 100% strictly typed.
@@ -27,81 +29,112 @@ npm install kaiwen-core-js
 
 ## Usage
 
-### 1. Initialization and Login
+### 1. Initialization
 
-Initialize the SDK by providing an App ID, the API URL of the Kaiwen Server, and your storage adapter.
+Initialize the SDK by providing your App ID, the API URL of the Kaiwen Server, and your storage adapter.
 
 ```typescript
 import { KaiwenApp, MemoryStorage } from "kaiwen-core-js";
 
 // Initialize the Facade
-const storage = new MemoryStorage(); // Or implement IKWStorage for SQLite/IndexedDB
+const storage = new MemoryStorage(); // Or implement IKWStorage for SQLite/AsyncStorage
 const app = new KaiwenApp(
   "core.kaiwen.notes",          // App ID (Isolated Encryption Domain)
-  "https://api.kaiwen.cloud",   // Kaiwen API Server
+  "http://localhost:8000",      // Kaiwen API Server
   "your_api_key_here",          // Server API Key
   storage
 );
+```
+
+### 2. Zero-Knowledge Authentication Flow
+
+#### A. Registration (Mnemonic Generation)
+Registering a new account automatically generates a client-side **12-word recovery mnemonic**. The mnemonic is encrypted locally using a key derived from the user's password, and the encrypted bundle is sent to the server. The raw mnemonic is returned so the user can back it up.
 
-// Login (Derives master key and isolated app key using PBKDF2 + HKDF)
-await app.login("apple banana cherry date elderberry fig grape hazelnut ice juice kiwi lemon");
+```typescript
+const registerRes = await app.register("username", "password");
+console.log("Account created!");
+console.log("Your 12-word recovery phrase is:", registerRes.mnemonic);
+// IMPORTANT: Prompt the user to write this down securely. It cannot be recovered by the server!
 ```
 
-### 2. Saving Encrypted Records
+#### B. Login with Password
+To log in on any device using just the password, the client requests the encrypted mnemonic from the server, decrypts it locally using the password-derived key, and derives the E2E Vault Key from the mnemonic.
+
+```typescript
+await app.loginWithPassword("username", "password");
+console.log("Logged in successfully! Vault is decrypted.");
+```
 
-Data is encrypted *before* it is saved to the local database queue.
+#### C. Direct Mnemonic Login (Passwordless)
+Users can log in directly using their 12-word recovery phrase. The client authenticates using a hash of the mnemonic (`mnemonic_hash`) and derives the vault keys directly.
 
 ```typescript
-// Save an end-to-end encrypted note
-const record = await app.saveRecord("user_123", { 
-  title: "My Secret Plan",
-  content: "This is fully encrypted using AES-256-GCM." 
-});
+await app.loginWithMnemonic("username", "amateur upper armed actor alert unit...");
+console.log("Logged in directly via mnemonic!");
+```
 
-console.log("Locally saved record ID:", record.id);
+#### D. Password Recovery (Forgotten Password)
+If a user forgets their password, they can reset it using their 12-word recovery mnemonic. The client sends a cryptographically signed proof (the mnemonic hash) to reset the password verifier on the server, re-encrypts the mnemonic under the new password key, and uploads it.
+
+```typescript
+await app.recoverPassword("username", "amateur upper armed actor alert...", "new_password_999");
+console.log("Password reset successfully! Vault unlocked with new password.");
 ```
 
-### 3. Pushing to the Cloud
+#### E. Password Rotation (Key Rotation)
+Changing the password re-encrypts the recovery mnemonic with the new password-derived key on the server. If the user was a legacy password-only user, the SDK automatically upgrades their account to E2EE mnemonic recovery and re-encrypts all local database records seamlessly.
 
-Once records are saved locally, push the pending queue to the Kaiwen Server.
+```typescript
+await app.rotatePassword("username", "old_password_123", "new_password_456");
+console.log("Password rotated and E2EE keys synchronized!");
+```
+
+### 3. Giriş ve Otomatik Canlı Eşitleme (startSync)
+
+Once logged in, call `startSync` to initialize real-time synchronization. It will pull the latest updates, connect the WebSocket channel, and trigger your callback whenever remote updates arrive.
 
 ```typescript
-const pushedCount = await app.pushSync();
-console.log(`Successfully synced ${pushedCount} records to the cloud.`);
+// Start real-time synchronization on startup or login
+await app.startSync(() => {
+  // Callback triggered when remote changes are received and synced locally
+  console.log("Database updated! Reloading UI...");
+  loadNotes();
+});
 ```
 
-### 4. Pulling from the Cloud
+### 4. Değişiklik Yapma (Otomatik Arkaplan Eşitleme)
 
-Pull changes made by other devices (e.g., a Python desktop app or an iOS device). The SDK will automatically handle decryption and logical clock conflict resolution.
+Creating, editing, or deleting a record automatically triggers a background push sync to the cloud. You do **not** need to call `pushSync()` manually.
 
 ```typescript
-const pulledCount = await app.pullSync();
-console.log(`Fetched ${pulledCount} new or updated records.`);
-
-// Decrypt and read the latest state
-const allRecords = await storage.getAllRecords();
-for (const r of allRecords) {
-    const decryptedPayload = await app.getDecryptedRecord(r.id);
-    console.log("Decrypted Data:", decryptedPayload);
-}
+// A. Yeni Kayıt Ekleme
+await app.saveRecord("user_123", { title: "New Note", content: "Secret data" }, "notes");
+
+// B. Kayıt Düzenleme (Edit)
+await app.updateRecord("note_id_123", { title: "Updated Note", content: "Modified secret data" });
+
+// C. Kayıt Silme (Delete)
+await app.deleteRecord("note_id_123");
 ```
 
----
+Whenever you perform any of these write actions, the library saves it locally and starts a background thread/promise to synchronize with the server, which broadcasts the change to all other active client instances instantly.
 
-## The Kaiwen Ecosystem Architecture
+### 5. Manuel Eşitleme (Opsiyonel)
 
-This library implements the precise directory mapping and modularity of the Python `kaiwen-core` SDK:
+If you need to force a manual sync, you can call push and pull directly:
 
-- `/core`: Exceptions and pipeline flows.
-- `/crypto`: Cryptographic engine (WebCrypto API).
-- `/models`: Database schemas, enums, and migration engines.
-- `/storage`: Vault (Key Management), Interfaces, and Chunked Asset Engine.
-- `/sync`: Network dispatchers.
-- `/utils`: Logging and telemetry.
+```typescript
+const pushed = await app.pushSync();
+const pulled = await app.pullSync();
+console.log(`Pushed: ${pushed}, Pulled: ${pulled}`);
+```
+
+---
 
 ## Security Notice
 
-- **Never hardcode mnemonics.**
+- **Never learn the user's raw mnemonic on the server.** The server only stores the E2E-encrypted mnemonic and the mnemonic hash for validation.
 - **Ensure TLS (HTTPS)** is used when communicating with the `kaiwen_server` to protect metadata (even though payloads are E2E encrypted).
 
 ---

package/dist/app.d.ts

@@ -1,6 +1,6 @@
 import { KWVault } from "./storage/vault";
 import { KWMigrationEngine } from "./models/migration";
-import { KWAssetEngine } from "./storage/assets";
+import { KWAssetEngine, KWAssetManifest } from "./storage/assets";
 import { KWPipeline } from "./core/pipeline";
 import { IKWStorage } from "./storage/db";
 import { KWRecord } from "./models/record";
@@ -11,11 +11,25 @@ export declare class KaiwenApp {
     assetEngine: KWAssetEngine;
     pipeline: KWPipeline;
     migration: KWMigrationEngine;
+    private ws;
     constructor(appId: string, apiUrl: string, apiKey: string, storage?: IKWStorage);
+    register(username: string, password: string): Promise<any>;
     login(mnemonic: string): Promise<void>;
-    saveRecord(userId: string, payload: any): Promise<KWRecord>;
+    loginWithMnemonic(username: string, mnemonic: string): Promise<void>;
+    loginWithPassword(username: string, password: string): Promise<void>;
+    recoverPassword(username: string, mnemonic: string, newPassword: string): Promise<any>;
+    logout(): Promise<void>;
+    saveRecord(userId: string, payload: any, category?: string): Promise<KWRecord>;
     getDecryptedRecord(id: string): Promise<any>;
+    getDecryptedRecordsByCategory(category: string): Promise<any[]>;
     deleteRecord(id: string): Promise<boolean>;
+    updateRecord(id: string, payload: any): Promise<boolean>;
+    rotatePassword(username: string, oldPassword: string, newPassword: string): Promise<void>;
     pushSync(): Promise<number>;
     pullSync(): Promise<number>;
+    uploadAsset(fileBuffer: Uint8Array, mimeType: string, userId: string): Promise<KWAssetManifest>;
+    downloadAsset(assetId: string, mimeType: string): Promise<Uint8Array>;
+    startSync(onSyncUpdate?: () => void): Promise<void>;
+    startSyncListener(onSyncUpdate?: () => void): void;
+    stopSyncListener(): void;
 }

package/dist/app.js

@@ -9,25 +9,79 @@ const pipeline_1 = require("./core/pipeline");
 const db_1 = require("./storage/db");
 const record_1 = require("./models/record");
 const validator_1 = require("./utils/validator");
+const logger_1 = require("./utils/logger");
 class KaiwenApp {
     constructor(appId, apiUrl, apiKey, storage = new db_1.MemoryStorage()) {
         this.appId = appId;
         this.storage = storage;
+        this.ws = null;
         this.vault = new vault_1.KWVault();
         this.assetEngine = new assets_1.KWAssetEngine(this.vault);
-        this.pipeline = new pipeline_1.KWPipeline(this.vault, storage, apiUrl, apiKey);
+        this.pipeline = new pipeline_1.KWPipeline(appId, this.vault, storage, apiUrl, apiKey);
         this.migration = new migration_1.KWMigrationEngine(storage, 1);
     }
+    async register(username, password) {
+        const mnemonic = engine_1.KWCryptoEngine.generateMnemonic();
+        const passwordKey = await engine_1.KWCryptoEngine.deriveMasterKeyFromPassword(password, username);
+        const encryptedMnemonic = await engine_1.KWCryptoEngine.encryptMnemonic(mnemonic, passwordKey);
+        const mnemonicHash = await engine_1.KWCryptoEngine.sha256(mnemonic);
+        const res = await this.pipeline.network.register(username, password, encryptedMnemonic, mnemonicHash);
+        res.mnemonic = mnemonic;
+        return res;
+    }
     async login(mnemonic) {
         await this.vault.login(mnemonic, this.appId);
         await this.migration.migrateAll();
     }
-    async saveRecord(userId, payload) {
+    async loginWithMnemonic(username, mnemonic) {
+        const mnemonicHash = await engine_1.KWCryptoEngine.sha256(mnemonic);
+        await this.pipeline.network.loginMnemonic(username, mnemonicHash, this.appId);
+        await this.vault.login(mnemonic, this.appId);
+        await this.migration.migrateAll();
+    }
+    async loginWithPassword(username, password) {
+        const authData = await this.pipeline.network.login(username, password, this.appId);
+        const encryptedMnemonic = authData.encrypted_mnemonic;
+        if (encryptedMnemonic) {
+            const passwordKey = await engine_1.KWCryptoEngine.deriveMasterKeyFromPassword(password, username);
+            const mnemonic = await engine_1.KWCryptoEngine.decryptMnemonic(encryptedMnemonic, passwordKey);
+            await this.vault.login(mnemonic, this.appId);
+        }
+        else {
+            await this.vault.loginWithPassword(password, username, this.appId);
+        }
+        await this.migration.migrateAll();
+    }
+    async recoverPassword(username, mnemonic, newPassword) {
+        const mnemonicHash = await engine_1.KWCryptoEngine.sha256(mnemonic);
+        const passwordKey = await engine_1.KWCryptoEngine.deriveMasterKeyFromPassword(newPassword, username);
+        const encryptedMnemonic = await engine_1.KWCryptoEngine.encryptMnemonic(mnemonic, passwordKey);
+        const res = await this.pipeline.network.recoverPassword(username, mnemonicHash, newPassword, encryptedMnemonic, this.appId);
+        await this.vault.login(mnemonic, this.appId);
+        await this.migration.migrateAll();
+        return res;
+    }
+    async logout() {
+        try {
+            await this.pipeline.network.logout();
+        }
+        catch (err) {
+            logger_1.KWLogger.info("Network logout failed, clearing token locally: " + err);
+            this.pipeline.network.setSessionToken(null);
+        }
+        finally {
+            this.vault.lock();
+            this.stopSyncListener();
+        }
+    }
+    async saveRecord(userId, payload, category = "default") {
         validator_1.KWValidator.validatePayload(payload);
         const encrypted = await engine_1.KWCryptoEngine.encryptPayload(payload, this.vault.getKey());
         const generateId = () => typeof crypto !== 'undefined' && crypto.randomUUID ? crypto.randomUUID() : Math.random().toString(36).substring(2);
-        const record = new record_1.KWRecord(generateId(), this.appId, userId, encrypted, Date.now() / 1000, 0, 1, 1);
+        const record = new record_1.KWRecord(generateId(), this.appId, userId, category, encrypted, Date.now() / 1000, 0, 0, 1, 1);
         await this.storage.saveRecord(record.id, record);
+        // Auto-trigger background pushSync
+        this.pushSync().catch(err => logger_1.KWLogger.error("Auto-push sync failed in saveRecord: " + err));
         return record;
     }
     async getDecryptedRecord(id) {
@@ -40,6 +94,21 @@ class KaiwenApp {
             return null;
         return await engine_1.KWCryptoEngine.decryptPayload(record.payload, this.vault.getKey());
     }
+    async getDecryptedRecordsByCategory(category) {
+        const all = await this.storage.getAllRecords();
+        const filtered = all.filter(r => r.category === category && r.is_deleted !== 1);
+        const decrypted = [];
+        for (const r of filtered) {
+            try {
+                const data = await engine_1.KWCryptoEngine.decryptPayload(r.payload, this.vault.getKey());
+                decrypted.push({ id: r.id, ...data });
+            }
+            catch (err) {
+                logger_1.KWLogger.error(`Decryption failed for record ${r.id}: ${err}`);
+            }
+        }
+        return decrypted;
+    }
     async deleteRecord(id) {
         const record = await this.storage.getRecord(id);
         if (!record)
@@ -49,13 +118,170 @@ class KaiwenApp {
         record.updated_at = Date.now() / 1000;
         record.logical_clock += 1;
         record.is_synced = 0;
-        return await this.storage.saveRecord(id, record);
+        const saved = await this.storage.saveRecord(id, record);
+        if (saved) {
+            this.pushSync().catch(err => logger_1.KWLogger.error("Auto-push sync failed in deleteRecord: " + err));
+        }
+        return saved;
+    }
+    async updateRecord(id, payload) {
+        const record = await this.storage.getRecord(id);
+        if (!record)
+            return false;
+        validator_1.KWValidator.validatePayload(payload);
+        const encrypted = await engine_1.KWCryptoEngine.encryptPayload(payload, this.vault.getKey());
+        record.payload = encrypted;
+        record.updated_at = Date.now() / 1000;
+        record.logical_clock += 1;
+        record.is_synced = 0;
+        const saved = await this.storage.saveRecord(id, record);
+        if (saved) {
+            this.pushSync().catch(err => logger_1.KWLogger.error("Auto-push sync failed in updateRecord: " + err));
+        }
+        return saved;
+    }
+    async rotatePassword(username, oldPassword, newPassword) {
+        logger_1.KWLogger.info("Initiating password rotation (Key Rotation)...");
+        if (!this.vault.isUnlocked()) {
+            throw new Error("Vault must be unlocked to perform password rotation.");
+        }
+        const oldKey = this.vault.getKey();
+        let mnemonic = this.vault.getMnemonic();
+        let wasLegacy = false;
+        if (!mnemonic) {
+            wasLegacy = true;
+            mnemonic = engine_1.KWCryptoEngine.generateMnemonic();
+        }
+        const newPasswordKey = await engine_1.KWCryptoEngine.deriveMasterKeyFromPassword(newPassword, username);
+        const newEncryptedMnemonic = await engine_1.KWCryptoEngine.encryptMnemonic(mnemonic, newPasswordKey);
+        // 1. Update password verifier on the server
+        await this.pipeline.network.changePassword(username, oldPassword, newPassword, newEncryptedMnemonic);
+        // 2. Derive new encryption key locally
+        const tempVault = new vault_1.KWVault();
+        await tempVault.login(mnemonic, this.appId);
+        const newKey = tempVault.getKey();
+        // 3. If the user was legacy, re-encrypt all local records
+        if (wasLegacy) {
+            logger_1.KWLogger.info("Upgrading legacy user to E2EE recovery: re-encrypting local records...");
+            const allRecords = await this.storage.getAllRecords();
+            for (const r of allRecords) {
+                if (r.category === "sync_metadata" || r.id.startsWith("__sync_metadata_")) {
+                    continue;
+                }
+                try {
+                    let decryptedPayload = null;
+                    if (r.payload) {
+                        decryptedPayload = await engine_1.KWCryptoEngine.decryptPayload(r.payload, oldKey);
+                    }
+                    if (decryptedPayload) {
+                        const newEncrypted = await engine_1.KWCryptoEngine.encryptPayload(decryptedPayload, newKey);
+                        r.payload = newEncrypted;
+                    }
+                    r.logical_clock += 1;
+                    r.updated_at = Date.now() / 1000;
+                    r.is_synced = 0;
+                    await this.storage.saveRecord(r.id, r);
+                }
+                catch (err) {
+                    logger_1.KWLogger.error(`Failed to rotate key for record ${r.id}: ${err}`);
+                }
+            }
+        }
+        // 4. Update current vault and session token
+        await this.vault.login(mnemonic, this.appId);
+        await this.pipeline.network.login(username, newPassword, this.appId);
+        // 5. Force push the re-encrypted records if they were upgraded
+        if (wasLegacy) {
+            logger_1.KWLogger.info("Pushing re-encrypted records to the cloud...");
+            await this.pushSync();
+        }
+        logger_1.KWLogger.info("Password rotation completed successfully!");
     }
     async pushSync() {
-        return await this.pipeline.pushSync(this.appId);
+        return await this.pipeline.pushSync();
     }
     async pullSync() {
-        return await this.pipeline.pullSync(this.appId);
+        return await this.pipeline.pullSync();
+    }
+    async uploadAsset(fileBuffer, mimeType, userId) {
+        const { manifest, encryptedChunks } = await this.assetEngine.encryptAsset(fileBuffer, mimeType);
+        for (let i = 0; i < encryptedChunks.length; i++) {
+            await this.pipeline.network.uploadAssetChunk({
+                asset_id: manifest.assetId,
+                chunk_index: i,
+                total_chunks: encryptedChunks.length,
+                payload: encryptedChunks[i]
+            });
+        }
+        await this.saveRecord(userId, manifest, "assets");
+        return manifest;
+    }
+    async downloadAsset(assetId, mimeType) {
+        const chunks = await this.pipeline.network.downloadAsset(assetId);
+        chunks.sort((a, b) => a.chunk_index - b.chunk_index);
+        const encryptedChunks = chunks.map(c => c.payload);
+        return await this.assetEngine.decryptAsset(encryptedChunks, mimeType);
+    }
+    async startSync(onSyncUpdate) {
+        logger_1.KWLogger.info("Initializing sync: performing initial pullSync...");
+        try {
+            await this.pullSync();
+        }
+        catch (e) {
+            logger_1.KWLogger.error("Initial pullSync failed during startSync: " + e);
+        }
+        this.startSyncListener(onSyncUpdate);
+        if (onSyncUpdate) {
+            onSyncUpdate();
+        }
+    }
+    startSyncListener(onSyncUpdate) {
+        if (this.ws) {
+            this.ws.close();
+        }
+        let wsUrl = this.pipeline.network["apiUrl"].replace(/^http/, "ws");
+        const token = this.pipeline.network["sessionToken"] || "";
+        wsUrl = `${wsUrl}/ws/${this.appId}?token=${token}`;
+        const WS = typeof WebSocket !== "undefined" ? WebSocket : globalThis.WebSocket;
+        if (!WS) {
+            logger_1.KWLogger.info("WebSocket is not supported in this environment.");
+            return;
+        }
+        this.ws = new WS(wsUrl);
+        this.ws.onmessage = async (event) => {
+            try {
+                const data = JSON.parse(event.data);
+                if (data.event === "SYNC_UPDATED") {
+                    logger_1.KWLogger.info("Real-time sync event received. Pulling updates...");
+                    await this.pullSync();
+                    if (onSyncUpdate) {
+                        onSyncUpdate();
+                    }
+                }
+                else if (data.event === "SESSION_EXPIRED") {
+                    logger_1.KWLogger.info("Session expired (password changed on another device). Logging out...");
+                    await this.logout();
+                    if (onSyncUpdate) {
+                        onSyncUpdate();
+                    }
+                }
+            }
+            catch (err) {
+                logger_1.KWLogger.error("Failed to process sync websocket message: " + err);
+            }
+        };
+        this.ws.onerror = (err) => {
+            logger_1.KWLogger.error("Sync WebSocket error: " + err);
+        };
+        this.ws.onclose = () => {
+            logger_1.KWLogger.info("Sync WebSocket closed.");
+        };
+    }
+    stopSyncListener() {
+        if (this.ws) {
+            this.ws.close();
+            this.ws = null;
+        }
     }
 }
 exports.KaiwenApp = KaiwenApp;

package/dist/core/pipeline.d.ts

@@ -1,10 +1,12 @@
 import { KWVault } from "../storage/vault";
+import { KWNetwork } from "../sync/network";
 import { IKWStorage } from "../storage/db";
 export declare class KWPipeline {
+    private appId;
     private vault;
     private storage;
-    private network;
-    constructor(vault: KWVault, storage: IKWStorage, apiUrl: string, apiKey: string);
-    pushSync(appId: string): Promise<number>;
-    pullSync(appId: string): Promise<number>;
+    network: KWNetwork;
+    constructor(appId: string, vault: KWVault, storage: IKWStorage, apiUrl: string, apiKey: string);
+    pushSync(): Promise<number>;
+    pullSync(): Promise<number>;
 }

package/dist/core/pipeline.js

@@ -5,12 +5,13 @@ const network_1 = require("../sync/network");
 const record_1 = require("../models/record");
 const logger_1 = require("../utils/logger");
 class KWPipeline {
-    constructor(vault, storage, apiUrl, apiKey) {
+    constructor(appId, vault, storage, apiUrl, apiKey) {
+        this.appId = appId;
         this.vault = vault;
         this.storage = storage;
         this.network = new network_1.KWNetwork(apiUrl, apiKey);
     }
-    async pushSync(appId) {
+    async pushSync() {
         logger_1.KWLogger.info("Starting pushSync...");
         const allRecords = await this.storage.getAllRecords();
         const pending = allRecords.filter(r => r.is_synced === 0);
@@ -23,25 +24,62 @@ class KWPipeline {
         }
         return syncedCount;
     }
-    async pullSync(appId) {
+    async pullSync() {
         logger_1.KWLogger.info("Starting pullSync...");
-        const allRecords = await this.storage.getAllRecords();
-        let latestUpdate = 0;
-        allRecords.forEach(r => {
-            if (r.updated_at > latestUpdate)
-                latestUpdate = r.updated_at;
-        });
-        const incomingRecords = await this.network.pull(appId, latestUpdate);
+        let lastPullTime = 0;
+        const metadata = await this.storage.getRecord(`__sync_metadata_${this.appId}`);
+        if (metadata) {
+            try {
+                const data = JSON.parse(metadata.payload);
+                lastPullTime = data.lastPullTime || 0;
+            }
+            catch (e) {
+                // ignore parsing errors
+            }
+        }
         let mergedCount = 0;
-        for (const remoteObj of incomingRecords) {
-            const remote = record_1.KWRecord.fromObject(remoteObj);
-            const local = await this.storage.getRecord(remote.id);
-            if (!local || remote.logical_clock > local.logical_clock) {
-                remote.is_synced = 1;
-                await this.storage.saveRecord(remote.id, remote);
-                mergedCount++;
+        let maxRemoteUpdatedAt = lastPullTime;
+        const limit = 50;
+        let hasMore = true;
+        let currentSince = lastPullTime;
+        while (hasMore) {
+            const querySince = (currentSince === lastPullTime)
+                ? Math.max(0, lastPullTime - 1)
+                : currentSince;
+            const incomingRecords = await this.network.pull(this.appId, querySince, limit);
+            if (incomingRecords.length === 0) {
+                hasMore = false;
+                break;
+            }
+            let batchMaxRemoteUpdatedAt = currentSince;
+            for (const remoteObj of incomingRecords) {
+                const remote = record_1.KWRecord.fromObject(remoteObj);
+                const local = await this.storage.getRecord(remote.id);
+                if (remote.updated_at > batchMaxRemoteUpdatedAt) {
+                    batchMaxRemoteUpdatedAt = remote.updated_at;
+                }
+                if (!local || remote.logical_clock > local.logical_clock) {
+                    remote.is_synced = 1;
+                    await this.storage.saveRecord(remote.id, remote);
+                    mergedCount++;
+                }
+            }
+            if (incomingRecords.length < limit) {
+                hasMore = false;
+            }
+            if (batchMaxRemoteUpdatedAt <= currentSince) {
+                hasMore = false;
+            }
+            else {
+                currentSince = batchMaxRemoteUpdatedAt;
+            }
+            if (batchMaxRemoteUpdatedAt > maxRemoteUpdatedAt) {
+                maxRemoteUpdatedAt = batchMaxRemoteUpdatedAt;
             }
         }
+        // Save metadata record
+        const metaRecord = new record_1.KWRecord(`__sync_metadata_${this.appId}`, this.appId, "system", "sync_metadata", JSON.stringify({ lastPullTime: maxRemoteUpdatedAt }), Date.now() / 1000, 1, 0, 1, 1);
+        await this.storage.saveRecord(metaRecord.id, metaRecord);
         return mergedCount;
     }
 }

package/dist/crypto/engine.d.ts

@@ -3,6 +3,14 @@ export declare class KWCryptoEngine {
     private static bufferToBase64;
     private static base64ToBuffer;
     static deriveAppKeyFromMnemonic(mnemonic: string, appId: string): Promise<CryptoKey>;
+    private static readonly WORDLIST;
+    static generateMnemonic(): string;
+    private static importAesKey;
+    static encryptMnemonic(mnemonic: string, masterKeyRaw: Uint8Array): Promise<string>;
+    static decryptMnemonic(encryptedMnemonic: string, masterKeyRaw: Uint8Array): Promise<string>;
+    static sha256(text: string): Promise<string>;
+    static deriveMasterKeyFromPassword(password: string, username: string): Promise<Uint8Array>;
+    static deriveAppKeyFromMasterKey(masterKeyRaw: Uint8Array, appId: string): Promise<CryptoKey>;
     static encryptPayload(payload: any, appKey: CryptoKey): Promise<string>;
     static decryptPayload(encryptedPayloadBase64: string, appKey: CryptoKey): Promise<any>;
     static encryptAssetChunk(data: Uint8Array, appKey: CryptoKey): Promise<string>;