memorio 4.7.3 → 4.8.0

package/README.md

@@ -67,6 +67,9 @@ npm i memorio
 
 # Optional - only if you use the React hook
 npm i react react-dom
+
+# Optional - only if you want a local sql.js for `sqlite` (default uses CDN)
+npm i sql.js
 ```
 
 ---
@@ -160,6 +163,88 @@ const user = await idb.data.get('my-db', 'users', 1)
 
 > IndexedDB is a browser-only primitive - see [Cross-Platform Behavior](#cross-platform-behavior) for what happens off the browser.
 
+### `sqlite` - in-memory SQLite, zero config
+
+`memorio.sqlite` is an optional, lazily-loaded SQLite engine backed by [`sql.js`](https://sql.js.org) (the optional `sql.js` npm package is **not** a build-time dependency). It runs entirely in memory — ideal for ad-hoc SQL, structured queryable data, and local relational lookups. The engine is fetched on first use and never loaded when you don't touch `sqlite`.
+
+```javascript
+import 'memorio' // or: import { sqlite } from 'memorio'
+
+await sqlite.ready                          // waits until the sql.js engine is loaded (or rejected)
+sqlite.db.support()                         // true when the current platform can run sql.js
+```
+
+Bases (databases) are addressed by name, like `idb`:
+const SQL = await sqlite.db.getSQL()        // the underlying sql.js engine (loaded once, cached)
+sqlite.db.version()                         // engine version string, or null before load
+```
+
+Bases (databases) are addressed by name, like `idb`. `sqlite.db.create('app')` opens (or creates) an in-memory database, while `sqlite.db.get('app')` retrieves an already-open handle (`sqlite.db.list/delete/size` round out the lifecycle):
+
+```javascript
+await sqlite.db.create('app')    // in-memory database handle
+await sqlite.query.run('app', 'CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)')
+``` `sqlite.data` gives a small shortcut layer, and `sqlite.query` runs SQL:
+
+```javascript
+// Write (shortcut over raw prepare/bind)
+await sqlite.data.set('app', 'users', { id: 1, name: 'Sara' })
+await sqlite.data.get('app', 'users', 1)    // { id: 1, name: 'Sara' }
+
+// SQL
+await sqlite.query.run('app', 'CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT)')
+const rows = await sqlite.query.select('app', 'SELECT * FROM items', [1])
+
+// Portability: export / import a whole base as a binary blob
+const blob = await sqlite.db.export('app')
+await sqlite.db.import('app', blob)
+
+// Clean up
+await sqlite.db.delete('app')
+```
+
+#### Loading the engine
+
+The default loader injects the sql.js UMD build (`sql-wasm-browser.js`) from the
+jsDelivr CDN as a classic `<script>`; on load it exposes the `initSqlJs` factory
+on `globalThis`, which memorio then calls with your `locateFile`. This avoids a
+bare `import('sql.js')` so bundlers never resolve the optional `sql.js`
+dependency at build time:
+
+```javascript
+// Zero config: CDN <script> loader (default)
+await sqlite.ready
+```
+
+If you prefer a local/bundled `sql.js` (offline, private network, or to pin a
+version), install it and supply your own loader:
+
+```bash
+npm i sql.js    # optional peer
+```
+
+```javascript
+import { sqlite } from 'memorio'
+
+sqlite.config({ loader: () => import('sql.js') })   // local/npm build of sql.js
+await sqlite.ready
+```
+
+You can also point the WASM `locateFile` step at your own CDN/base via
+`sqliteWasmBase` on `globalThis`, or pass a custom `locateFile` through
+`sqlite.config({ locateFile })`:
+
+```javascript
+globalThis.sqliteWasmBase = 'https://your-cdn.example.com/sql.js/dist/'
+sqlite.config({ locateFile: (file) => `https://your-cdn.example.com/sql.js/dist/${file}` })
+```
+
+When the engine can't load (e.g. no WASM support), `sqlite.ready` rejects, `sqlite._disabled` becomes `true`, and `sqlite._warning` explains why — `db.support()` simply returns `false` so you can degrade gracefully.
+
+- `sqlite.config({ persistence: true })` / `sqlite.db.create('app', { persistence: true })` snapshot a database to `store` (localStorage, namespaced) and restore it on reopen — sql.js databases are in-memory by default; see [SQLite docs – Persistence & dev download](markdown/SQLITE.md#example-5-persistence--dev-download).
+
+See [SQLite docs](markdown/SQLITE.md) for the full reference.
+
 ### `observer` / `useObserver` - watch a path, react to it
 
 ```javascript
@@ -403,6 +488,126 @@ See [Memory docs](markdown/MEMORY.md) for full reference.
 
 ---
 
+## Synchronization & Cloud (optional)
+
+`memorio.memory` is **local-first**. The data is created and served from the
+device; the cloud is only ever a **transport/persistence provider**, never the
+source of truth. Enabling sync does not replace local storage — it *mirrors* it.
+
+```
+                    memorio
+                       │
+              ┌────────┴────────┐
+              │   Memory Engine │
+              └────────┬────────┘
+          ┌────────────┼────────────┐
+          ▼            ▼            ▼
+        local      SQLite         cloud
+        memory     durable        sync
+```
+
+### Configuring a backend
+
+Sync is **opt-in**. You supply an application-owned `provider` that knows how to
+talk to your backend (REST, WebSocket, Supabase, a custom agent server, a PostgreSQL
+database, etc.). **Memorio never handles credentials** — authentication and
+authorization live in your backend/provider (OWASP A01: Broken Access Control):
+
+```ts
+memorio.memory.configure({
+  namespace: 'user:123:device:abc',   // tenant/user/device — partitions the journal
+  provider: {                          // application-owned transport
+    push(ops) { return fetch('/api/sync', { method: 'POST', body: JSON.stringify(ops), headers: auth }) }
+    pull(since) { return fetch(`/api/sync?since=${since}`).then(r => r.json()) }
+  },
+  auto: true                           // auto-replay pending ops on focus/online (default true)
+})
+
+await memorio.memory.ready   // waits until the local journal substrate is chosen
+```
+
+Once configured, `memorio.memory` records every operation locally and keeps it
+until the provider acknowledges it.
+
+### The local journal
+
+Every mutation (`remember`, `update`, `forget`, `expire`, `confirm`, `supersede`)
+is written to a local, **namespaced** operation journal:
+
+```ts
+await memory.remember('user.language', 'Italian', { scope: 'local' })
+
+memory.journal.pending()      // -> [{ id, key:'user.language', value:'Italian', operation:'remember', sync:'pending', version: 1, updatedAt: ... }]
+memory.journal.markSynced([id])
+memory.journal.replay()       // pushes pending() to provider.provider, marks synced, optional pull+apply
+memory.journal.status()       // 'sqlite' | 'store'  (substrate in use)
+```
+
+The journal stores the full entry plus `operation`, `version`, `updatedAt` and
+`sync` status, so conflicts can be resolved when connectivity returns.
+
+### Substrates
+
+The journal is persisted on `store` (localStorage / in-memory Map fallback) — **not** on a sql.js database, because sql.js databases are volatile (in-memory) and would lose pending operations on reload:
+
+| Substrate | Used for | Notes |
+|---|---|---|
+| `store` (localStorage) | sync journal | persistent across reloads; namespaced; Map fallback in Node |
+| `sqlite` (sql.js) | ad-hoc SQL / value storage | in-memory by default — use `persistence: true` to snapshot to `store` |
+| `idb` (IndexedDB) | `memorio.memory` durable scope values | persistent; persistent across reloads |
+
+The `sqlite.db.download(name)` dev helper also triggers a browser `.sqlite` download of an in-memory database on demand.
+
+### Sync = operations, not database dumps
+
+Memorio syncronizza **operazioni di memoria**, never a raw database dump. When
+two devices diverge, conflict resolution is based on `confidence`,
+`lastConfirmedAt`, `source`, `version` and `scope` — not just "last write wins":
+
+```ts
+// Device A: user.language = Italian, confidence 0.92
+// Device B: user.language = English, confidence 0.61
+// → the higher-confidence entry wins locally; the provider decides for shared.
+```
+
+The `namespace` (tenant / user / device) is the single partition key: local
+journal reads and writes are scoped to it, and they can never cross namespaces —
+a client holding a fake/forged namespace simply sees its own empty journal.
+
+### Scopes (isolation, not a security boundary)
+
+```
+scope: 'device'   // only this browser/device
+scope: 'user'      // follows the user across devices (via sync)
+scope: 'shared'   // shared across users/tenant
+```
+
+`scope: 'user'`/`'shared'` require a provider + namespace. `scope: 'device'`
+is local only. As with `memorio.createContext`, **scoping is a naming convention,
+not a security boundary** — enforce real isolation in your backend.
+
+### Security posture
+
+- **NIST SP 800-53 / OWASP**: no credentials, tokens, or secrets are read from or
+  stored by `memorio`; sensitive state you place in `state`/`store`/`session`/`idb`
+  is not encrypted by memorio (see [Security](#security)).
+- **Namespace isolation**: journal reads/writes are keyed by
+  `namespace:id` at the storage layer; there is no API to enumerate or open
+  another namespace's journal (defense-in-depth).
+- **No dynamic code**: journal entries are strictly JSON-round-tripped and
+  size-capped (10 MB/entry); no `eval`/template-injection of provider data.
+- **Trust boundary**: the provider/backend owns authentication, authorization,
+  and remote-side conflict resolution. Memorio owns the local durable copy and
+  the operation log; it surfaces `conflict`/`error` rows via `journal.pending()`.
+- **NSA/CISA advice (data-at-rest/secrets)**: if you persist `state`/`store`
+  server-side or ship user data through your backend, encrypt it server-side with
+  keys you manage; memorio treats the local journal as untrusted-from-the-browser
+  and does not attest its own integrity.
+
+See [SQLite docs](markdown/SQLITE.md), [Memory docs](markdown/MEMORY.md), and the [Synchronization & Cloud guide](markdown/SYNC.md).
+
+---
+
 ## Cross-Platform Behavior
 
 memorio runs everywhere JavaScript does - but "everywhere" means different guarantees in different places, and we'd rather tell you now than have you find out at 2am:
@@ -415,6 +620,7 @@ memorio runs everywhere JavaScript does - but "everywhere" means different guara
 | `store` | `localStorage` | `Map` fallback - **not durable across restarts** | `Map` fallback - not durable | `localStorage` where available, else `Map` |
 | `session` | `sessionStorage` | `Map` fallback - not durable | `Map` fallback - not durable | `sessionStorage` where available, else `Map` |
 | `idb` | ✅ `IndexedDB` | ❌ not available | ❌ not available | ⚠️ check `getCapabilities()` |
+| `sqlite` | ✅ `sql.js` (lazy, CDN by default) | ❌ not available | ❌ not available | ⚠️ check `sqlite.db.support()` |
 | `devtools` | ✅ | ❌ | ❌ | ⚠️ |
 
 Same API top to bottom - that's the promise. But if your server code leans on `store.get(...)` surviving a redeploy, know that on Node/Deno it won't; the fallback is an in-memory cache with the same shape, not durable storage.

package/SUMMARY.md

@@ -12,6 +12,7 @@
 * [Store](markdown/STORE.md) - Persistent localStorage management
 * [Session](markdown/SESSION.md) - Temporary sessionStorage management
 * [IDB](markdown/IDB.md) - IndexedDB for large data storage
+* [SQLite](markdown/SQLITE.md) - SQLite in the browser via sql.js (WASM)
 
 ## Typed & Validated
 
@@ -26,6 +27,12 @@
 ## Memory System
 
 * [Memory](markdown/MEMORY.md) - Semantic memory layer with TTL, confidence, scopes
+* [Node Attachment](markdown/MEMORY-ATTACHMENT.md) - Dynamic node attachment system
+
+## Experimental
+
+* [Memory Substrates Benchmark](experimental/memory-substrates/BENCHMARK.md) - Protocol for evaluating runtime substrates
+* [Leak Detection](experimental/memory-substrates/LEAK-DETECTION.md) - Tooling & methods for memory leak analysis
 
 ## Platform & Compatibility