memorio 4.7.3 → 4.8.0

package/markdown/SYNC.md

@@ -0,0 +1,170 @@
+# Synchronization & Cloud (optional)
+
+`memorio.memory` is **local-first**. Data is created and served from the device;
+the cloud is only ever a **transport/persistence provider**, never the source of
+truth. Enabling sync does not replace local storage — it *mirrors* it.
+
+```
+                    memorio
+                       │
+              ┌────────┴────────┐
+              │   Memory Engine │
+              └────────┬────────┘
+          ┌────────────┼────────────┐
+          ▼            ▼            ▼
+        local      SQLite         cloud
+        memory     durable        sync
+```
+
+## 1. The rule: the data is born local
+
+```ts
+memorio.memory.remember('user.language', 'Italian', { scope: 'local' })
+//        ↓ local first
+//   store / sessionStorage / IndexedDB / sql.js
+//        ↓ sync / push (when online)
+//   cloud provider
+```
+
+The cloud therefore does not **replace** memory: it **replicates** it. This
+gives you: offline-first, lowest latency, data available immediately,
+synchronization when online, multi-device, multi-user, centralized persistence.
+
+We deliberately do **not** provide:
+
+```ts
+// ❌ two mental models
+memory.cloud.save(...)
+```
+
+Instead:
+
+```ts
+memorio.memory.remember('user.language', 'Italian')
+// and a single configuration point:
+memorio.memory.configure({ sync: { provider: myCloudProvider, namespace: '…' } })
+```
+
+## 2. Scopes (isolation, not a security boundary)
+
+| Scope | Lifetime | Syncs by default |
+|---|---|---|
+| `'device'` | this browser/device only | no (sticky) |
+| `'user'` | follows the user across devices | yes (requires provider + namespace) |
+| `'shared'` | shared across users / tenant | yes (requires provider + namespace) |
+
+> As with `memorio.createContext`, **scoping is a naming convention, not a
+> security boundary.** Enforce real isolation server-side.
+
+## 3. SQLite as the local durable store
+
+SQLite (`sql.js`) is **in-memory by default** (volatie per page load). It becomes
+the durable journal/value store when you opt in:
+
+- `sqlite.config({ persistence: true })` / `sqlite.db.create('app', { persistence: true })`
+  snapshot the database to `store` (localStorage) and restore it on reopen.
+- Writes are snapshotted via sql.js `updateHook` (debounced).
+- `sqlite.db.persist(name)` forces an immediate save; `sqlite.db.close(name)`
+  flushes + closes; `sqlite.db.download(name, file?)` triggers a browser
+  `.sqlite` download (dev convenience).
+
+See `docs/markdown/SQLITE.md` for the full SQLite reference.
+
+## 4. The local operation journal
+
+The **sync journal** is the durable op log that drives cloud reconciliation.
+It is persisted on `store` (localStorage) — **not** on an in-memory sql.js db,
+because pending operations must survive a refresh for offline-first to work.
+
+| Method | Returns | Notes |
+|---|---|---|
+| `memory.journal.append(entry, operation)` | `Promise<MemoryEntry>` | records `remember\|update\|forget\|expire\|confirm\|supersede` with `sync:'pending'` |
+| `memory.journal.pending()` | `Promise<MemoryEntry[]>` | rows where `sync != 'synced'`, for the current namespace |
+| `memory.journal.markSynced(ids)` | `Promise<number>` | advances rows to `synced` (namespace-scoped) |
+| `memory.journal.get(id)` | `Promise<MemoryEntry \| null>` | single entry, namespace-scoped |
+| `memory.journal.clear()` | `Promise<void>` | wipes the current namespace's journal |
+| `memory.journal.replay()` | `Promise<SyncAck>` | pushes `pending()` to the provider, marks synced, optional `pull` |
+| `memory.journal.status()` | `Promise<'store'>` | the substrate in use |
+
+We sync **operations of memory**, never a raw database dump:
+
+```
+user A        device A
+remember X ─────► local ─────► sync ─────► cloud
+forget Z ──────► local ─────► sync ─────► cloud
+```
+
+## 5. Conflict resolution
+
+The cloud must not simply say "last write wins." Memorio tags every entry with:
+
+- `confidence` (0–1, user/system trust in the value)
+- `lastConfirmedAt` / `updatedAt` (epoch ms)
+- `version` (monotonic per-key counter)
+- `source` / `scope`
+
+Remote conflicts are surfaced as `sync:'conflict'` rows via
+`journal.pending()`; the provider's `resolve(op)` hint decides locally. Example:
+
+```
+Laptop:  language=Italian, confidence=0.92
+Phone:   language=English, confidence=0.61
+→ higher-confidence entry wins locally; the provider decides for shared scope.
+```
+
+## 6. Configuring a backend
+
+Sync is **opt-in**. You supply an application-owned `provider` that knows how to
+talk to your backend (REST, WebSocket, Supabase, a custom agent server, …).
+
+```ts
+memorio.memory.configure({
+  namespace: 'user:123:device:abc',   // tenant/user/device — partitions the journal
+  provider: {
+    push(ops) { return fetch('/api/sync', { method: 'POST', body: JSON.stringify(ops), headers: authHeaders }) }
+    pull(since) { return fetch(`/api/sync?since=${since}`).then(r => r.json()) }
+    resolve(op) { return op.confidence >= 0.8 ? 'local' : 'remote' }
+  },
+  auto: true                           // auto-replay on focus/online (default true)
+})
+```
+
+```ts
+interface SyncProvider {
+  push(ops: MemoryEntry[]): Promise<{ synced: string[]; conflicts?: string[]; error?: string }>
+  pull?(since?: number): Promise<MemoryEntry[]>
+  resolve?(op: MemoryEntry): Promise<'local' | 'remote' | 'merge'>
+}
+```
+
+`memorio.memory.ready` resolves once the local journal substrate is chosen.
+
+## 7. Security (NIST / OWASP / NSA posture)
+
+- **Memorio never handles credentials.** No passwords, tokens, or API keys are
+  read from or stored by memorio. Authentication/authorization live in your
+  `provider`/backend (OWASP A01: Broken Access Control).
+- **Namespace isolation.** The journal is keyed by `namespace:id` at the storage
+  layer; there is **no API** to enumerate or open another namespace's journal. A
+  client holding a forged/fake namespace simply sees its own (empty) journal.
+- **No dynamic code.** Journal entries are strictly JSON-round-tripped,
+  size-capped (10 MB/entry), and never `eval`'d. The sql.js loader never
+  `import()`s a bare specifier that could be hijacked at build time.
+- **Trust boundary:** memorio owns the local durable copy + operation log; the
+  provider/backend owns remote-side auth and conflict resolution. Memorio
+  surfaces `conflict`/`error` rows; it does not fabricate a winner.
+- **Data-at-rest (NSA/CISA).** memorio's `store`/`idb`/`sqlite` snapshots are
+  **not encrypted**. If you persist user data server-side or ship it through your
+  backend, encrypt it server-side with keys you manage — memorio treats the local
+  store as untrusted-from-the-browser and does not attest its own integrity.
+
+## 8. Where data lives
+
+| Substrate | API | Volatile? | Persistent? |
+|---|---|---|---|
+| in-memory `Proxy` | `state` | yes (per tab) | no |
+| `localStorage` / Map | `store` | no | yes (browser) |
+| `sessionStorage` / Map | `session` | no | per-tab (browser) |
+| IndexedDB | `idb`, `memory` durable | no | yes |
+| sql.js (WASM heap) | `sqlite` | **yes** | only with `persistence: true` (snapshot → `store`) |
+| sync journal | `memory.journal` | no | yes (`store`) |

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "memorio",
   "codeName": "memorio",
-  "version": "4.7.3",
+  "version": "4.8.0",
   "description": "Memorio, State + Observer, Store and iDB for an easy life - Cross-platform compatible",
   "main": "./index.cjs",
   "browser": "./index.js",
@@ -34,8 +34,9 @@
     "reactive",
     "universal",
     "cross-platform",
-    "node",
-    "deno",
+    "sqlite",
+    "sql.js",
+    "wasm",
     "browser",
     "frontend"
   ],
@@ -77,6 +78,11 @@
   "overrides": {
     "esbuild": "^0.25.0"
   },
+  "peerDependenciesMeta": {
+    "sql.js": {
+      "optional": true
+    }
+  },
   "exports": {
     ".": {
       "import": "./index.js",

package/types/exports.d.ts

@@ -17,6 +17,7 @@
 /// <reference path="./session.d.ts" />
 /// <reference path="./cache.d.ts" />
 /// <reference path="./idb.d.ts" />
+/// <reference path="./sqlite.d.ts" />
 /// <reference path="./observer.d.ts" />
 /// <reference path="./useObserver.d.ts" />
 
@@ -26,6 +27,7 @@ export const store: _store
 export const session: _session
 export const cache: _cache
 export const idb: _idb
+export const sqlite: _sqlite
 export const observer: _observer
 export const useObserver: _useObserver
 export const dispatch: _dispatch

package/types/memorio.d.ts

@@ -1,5 +1,11 @@
 import type {
   MemoryEntry,
+  MemorySync,
+  MemoryOperation,
+  SyncProvider,
+  SyncConfig,
+  SyncAck,
+  SyncDirection,
   RememberOptions,
   RecallOptions,
   ContextOptions,
@@ -107,6 +113,29 @@ interface MemoryAPI {
   stats(): Promise<MemoryStats>
   forgetExpired(): Promise<number>
   clear(): Promise<void>
+  /**
+   * Configure the optional cloud-sync layer. Supplying a `provider` (an
+   * application-owned object that knows how to talk to your backend) enables
+   * the local operation journal. The `namespace` (tenant/user/device)
+   * partitions the journal so contexts/tenants cannot read each other's
+   * operations. Memorio never handles credentials — auth lives in the
+   * provider/backend.
+   */
+  configure(opts: SyncConfig): MemoryAPI
+  /** Local operation journal. Writes are namespaced and durable (SQLite when
+   * available, else localStorage). Use `replay()` to push `pending()` entries
+   * to the configured provider. */
+  journal: MemoryJournal
+}
+
+interface MemoryJournal {
+  append(entry: MemoryEntry, operation: MemoryOperation): Promise<MemoryEntry>
+  pending(): Promise<MemoryEntry[]>
+  markSynced(ids: string[]): Promise<number>
+  get(id: string): Promise<MemoryEntry | null>
+  clear(): Promise<void>
+  replay(): Promise<SyncAck>
+  status(): Promise<'store'>
 }
 
 type memorio = _memorio

package/types/memory.d.ts

@@ -9,6 +9,55 @@ export type MemoryType =
 
 export type MemoryStatus = 'active' | 'obsolete' | 'superseded'
 
+/**
+ * Logical operations recorded in the local sync journal.
+ * The cloud side only ever sees operations — never a raw dump of a substrate.
+ */
+export type MemoryOperation =
+  | 'remember'
+  | 'update'
+  | 'forget'
+  | 'expire'
+  | 'confirm'
+  | 'supersede'
+
+/**
+ * Synchronization state of a journal entry against a remote/cloud backend.
+ */
+export type MemorySync = 'pending' | 'synced' | 'conflict' | 'error'
+
+export type SyncDirection = 'up' | 'down' | 'both'
+
+/**
+ * Pluggable cloud/sync transport. Memorio never ships a credential flow:
+ * the application supplies a `provider` that knows how to talk to its backend
+ * (REST, WebSocket, Supabase, a custom agent server, ...). Memorio owns the
+ * local journal, the provider only moves operations.
+ */
+export interface SyncProvider {
+  /** Pushes pending local operations; returns ids that were accepted */
+  push(ops: MemoryEntry[]): Promise<{ synced: string[]; conflicts?: string[]; error?: string }>
+  /** Optional pull of remote operations newer than `since` (epoch ms) */
+  pull?(since?: number): Promise<MemoryEntry[]>
+  /** Optional conflict resolution hint: 'local' | 'remote' | 'merge' */
+  resolve?(op: MemoryEntry): Promise<'local' | 'remote' | 'merge'>
+}
+
+export interface SyncConfig {
+  /** Sync provider implementation (application supplied) */
+  provider?: SyncProvider
+  /** Which direction to sync; 'both' is the default */
+  direction?: SyncDirection
+  /** Auto-sync on every local write? defaults to true */
+  auto?: boolean
+}
+
+export interface SyncAck {
+  synced: string[]
+  conflicts?: string[]
+  error?: string
+}
+
 export interface MemoryEntry<T = any> {
   id: string
   key: string
@@ -23,6 +72,14 @@ export interface MemoryEntry<T = any> {
   createdAt: number
   lastConfirmedAt: number
   supersededId?: string | null
+  /** Monotonic per-key write counter, used for last-writer / conflict resolution */
+  version?: number
+  /** Epoch ms of the last mutation that touched this entry */
+  updatedAt?: number
+  /** Last logical operation that produced this entry */
+  operation?: MemoryOperation
+  /** Local-cloud sync state of the entry (undefined => not tracked for sync) */
+  sync?: MemorySync
 }
 
 export interface RememberOptions<T = any> {

package/types/sqlite.d.ts

@@ -0,0 +1,35 @@
+/*!
+  memorio
+  Copyright (c) 2019 Dario Passariello <dariopassariello@gmail.com>
+  Licensed under MIT License, see
+  dario.passariello.ca
+*/
+
+/**
+ * Memorio SQLite module.
+ *
+ * Backed by the optional `sql.js` package (SQLite compiled to WebAssembly).
+ * Lazily loaded in the browser; disabled in non-browser environments.
+ */
+interface _sqlite {
+  /** Database connection tools (create, get, delete, list, size, export, import). */
+  db: any
+  /** SQL execution tools (run, select). */
+  query: any
+  /** CRUD shortcut helpers (set, get). */
+  data: any
+  /** Lazy-load / configure the sql.js engine (wasm loader, locateFile, etc.). */
+  config: (opts: any) => any
+  /** Promise that resolves once the sql.js engine has been initialized. */
+  ready: Promise<void> | null
+  /** True when running outside a supported browser environment. */
+  _disabled?: boolean
+  /** Human readable reason the module is disabled, when applicable. */
+  _warning?: string
+  [key: string]: any
+}
+
+declare var sqlite: _sqlite
+type sqlite = _sqlite
+
+declare var sqlbases: any[]