memorio 4.9.35 → 5.0.0

package/markdown/STATE.md

@@ -0,0 +1,160 @@
+> **Status:** Published
+> **Date:** 2026-09-12
+> **Deciders:** Memorio 5.x Core Team
+> **Scope**: API Reference
+> **Standard**: Memorio API Specification v5
+>
+---
+# State - Memorio
+
+> ✅ **Universal**: Works in Browser, Node.js, Deno, and Edge Workers
+
+State is a reactive global state manager using JavaScript Proxies. It's simple, powerful, and requires no setup. Data persists only in memory during the session.
+
+## Installation
+
+```bash
+npm install memorio
+```
+
+```javascript
+import { state } from 'memorio';
+```
+
+That's it. `state` is ready to use.
+
+> **Classic `import`**: `state` is also available as a named export from the global entrypoint.
+> `import 'memorio/global'` exposes the same proxy as `globalThis.state`.
+
+---
+
+## Quick Examples
+
+### Example 1: Basic Usage
+
+```javascript
+// Set a value
+state.name = 'Mario';
+state.age = 25;
+
+// Get a value
+console.debug(state.name); // "Mario"
+
+// Simple object
+state.user = { name: 'Luigi', level: 1 };
+```
+
+### Example 2: Intermediate
+
+```javascript
+// Array operations
+state.items = [1, 2, 3];
+state.items.push(4);
+console.debug(state.items); // [1, 2, 3, 4]
+
+// Nested objects
+state.config = { theme: 'dark', lang: 'en' };
+state.config.theme = 'light';
+
+// List all states
+console.debug(state.list);
+```
+
+### Example 3: Advanced
+
+```javascript
+// Lock state to prevent modifications
+state.frozenConfig = { maxUsers: 100 };
+state.frozenConfig.lock();
+// Now state.frozenConfig cannot be modified
+
+// Path tracking
+const path = state.user.path;
+console.debug(path.name); // "user"
+console.debug(path.profile.name); // "user.profile"
+
+// Get full path as string
+console.debug(state.user.__path); // "state.user"
+
+// Protected keys (internal use)
+console.debug(protect); // Array of protected keys
+```
+
+---
+
+## API Reference
+
+### Properties
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `state.list` | Array | Get all current state keys (deep copy) |
+| `state.path` | Object | Get path tracker for current location |
+| `state.__path` | string | Get full path as string |
+
+### Methods
+
+| Method | Parameters | Description |
+|--------|------------|-------------|
+| `state.remove(key)` | `key: string` | Remove a specific state |
+| `state.removeAll()` | none | Clear all states |
+
+### Lock
+
+```javascript
+// Lock an object or array
+state.myArray = [1, 2, 3];
+state.myArray.lock();
+
+// Now any modification will fail
+state.myArray.push(4); // Error: state 'myArray' is locked
+```
+
+---
+
+## How It Works
+
+Memorio uses JavaScript `Proxy` to intercept get/set operations on the global `state` object. This allows:
+
+1. **Reactivity** - Any change can trigger observers
+2. **Nested objects** - Deep path tracking
+3. **Type safety** - Full TypeScript support
+
+---
+
+## Platform Notes
+
+| Platform | Support | Notes |
+|----------|---------|-------|
+| Browser | ✅ Full | In-memory, lost on refresh |
+| Node.js | ✅ Full | In-memory, lost on restart |
+| Deno | ✅ Full | In-memory, lost on restart |
+| Edge Workers | ✅ Full | In-memory, lost on function cold start |
+
+**Note**: In server environments (Node.js/Deno), use `memorio.createContext()` for request isolation.
+
+---
+
+## Best Practices
+
+1. Use descriptive keys: `state.userProfile` not `state.up`
+2. Group related data: `state.cart.items` not `state.cartItems`
+3. Lock static config: `state.appConfig.lock()`
+4. Clean up on logout: `state.removeAll()`
+5. Use path tracking for debugging: `state.myData.__path`
+
+---
+
+## Common Errors
+
+```javascript
+// Error: protected key
+state._internal = 'value';
+// Output: "key _internal is protected"
+
+// Error: locked state
+state.locked = { x: 1 };
+state.locked.lock();
+state.locked.x = 2;
+// Output: "Error: state 'locked' is locked"
+```

package/markdown/STORE.md

@@ -0,0 +1,171 @@
+> **Status:** Published
+> **Date:** 2026-09-12
+> **Deciders:** Memorio 5.x Core Team
+> **Scope**: API Reference
+> **Standard**: Memorio API Specification v5
+>
+---
+# Store - Memorio
+
+> 🖥️ **Browser & Edge**: Uses localStorage for persistence
+> ⚙️ **Node.js/Deno**: Falls back to in-memory storage (not persistent)
+
+Store provides persistent localStorage management with a simple API. Data survives page refreshes and browser restarts.
+
+## Installation
+
+```bash
+npm install memorio
+```
+
+```javascript
+import { store } from 'memorio';
+```
+
+> **Classic `import`**: `store` is also available via the global entrypoint.
+> `import 'memorio/global'` exposes the same instance as `globalThis.store`.
+
+---
+
+## Quick Examples
+
+### Example 1: Basic Usage
+
+```javascript
+// Save data
+store.set('username', 'Mario');
+store.set('score', 1500);
+
+// Read data
+console.debug(store.get('username')); // "Mario"
+console.debug(store.get('score'));    // 1500
+
+// Check if using real persistence
+console.debug(store.isPersistent); // true in browser, false in Node.js/Deno
+```
+
+### Example 2: Intermediate
+
+```javascript
+// Store objects
+store.set('user', { name: 'Luigi', level: 5 });
+const user = store.get('user');
+console.debug(user.name); // "Luigi"
+
+// Remove single item
+store.remove('username');
+
+// Check size
+const totalSize = store.size();
+console.debug(`${totalSize} bytes`);
+```
+
+### Example 3: Advanced
+
+```javascript
+// Get storage quota (returns Promise<[usage, quota]> in KB)
+const [used, total] = await store.quota();
+console.debug(`Using ${used} out of ${total} KB`);
+
+// Get total size in characters
+const size = store.size();
+console.debug(`${size} bytes`);
+
+// Clear all data
+store.removeAll();
+// or use alias
+store.clearAll();
+
+// Handle errors gracefully
+try {
+  store.set('largeData', hugeObject);
+} catch (err) {
+  console.error('Storage full:', err);
+}
+```
+
+---
+
+## API Reference
+
+### Methods
+
+| Method | Parameters | Returns | Description |
+|--------|------------|---------|-------------|
+| `store.get(name)` | `name: string` | `any` | Get value from storage |
+| `store.set(name, value)` | `name: string, value: any` | `void` | Save value to storage |
+| `store.remove(name)` | `name: string` | `boolean` | Remove single item |
+| `store.delete(name)` | `name: string` | `boolean` | Alias for remove |
+| `store.removeAll()` | none | `boolean` | Clear all storage |
+| `store.clearAll()` | none | `boolean` | Alias for removeAll |
+| `store.size()` | none | `number` | Get total size in characters |
+| `store.quota()` | none | `Promise<[number, number]>` | Get storage usage/quota in KB |
+
+### Properties
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `store.isPersistent` | `boolean` | `true` if using real localStorage, `false` if in-memory fallback |
+
+### Supported Types
+
+```javascript
+// All JSON-serializable types work
+store.set('string', 'hello');
+store.set('number', 42);
+store.set('boolean', true);
+store.set('array', [1, 2, 3]);
+store.set('object', { key: 'value' });
+store.set('null', null);
+store.set('undefined', null); // converted to null
+```
+
+### Not Supported
+
+```javascript
+// Functions will log an error
+store.set('myFunc', () => {});
+// Output: "It's not secure to store functions."
+```
+
+---
+
+## Platform Comparison
+
+| Feature | Store | Session | Cache | IDB |
+|---------|-------|---------|-------|-----|
+| **Storage** | localStorage | sessionStorage | Memory | IndexedDB |
+| **Lifetime** | Forever | Until tab closes | Until refresh | Forever |
+| **Capacity** | ~5-10 MB | ~5-10 MB | Unlimited | 50+ MB |
+| **Platform** | Browser/Edge | Browser/Edge | All | Browser |
+| **Persistence** | ✅ true | N/A | ❌ false | ✅ true |
+
+---
+
+## How It Works
+
+Store wraps the browser's `localStorage` API with:
+
+- Automatic JSON serialization/deserialization
+- Error handling for parse failures
+- Size calculation
+- Quota monitoring
+
+---
+
+## Storage Limits
+
+- **Chrome/Safari**: ~5-10 MB
+- **Firefox**: ~10 MB
+- **Edge**: ~5-10 MB
+
+Use `store.quota()` to monitor usage.
+
+---
+
+## Best Practices
+
+1. Prefix keys: `store.set('app_username', '...')`
+2. Check before set: `if (store.get('key')) { ... }`
+3. Handle quota: Try/catch around large data
+4. Clean up: `store.removeAll()` on logout

package/markdown/SYNC.md

@@ -0,0 +1,319 @@
+> **Status:** Published
+> **Date:** 2026-09-12
+> **Deciders:** Memorio 5.x Core Team
+> **Scope**: API Reference
+> **Standard**: Memorio API Specification v5
+>
+---
+# Synchronization & Cloud (optional)
+
+`memorio.memory` is **local-first**. Data is created and served from the device;
+the cloud is only ever a **transport/persistence provider**, never the source of
+truth. Enabling sync does not replace local storage - it *mirrors* it.
+
+```
+                    memorio
+                       │
+              ┌────────┴────────┐
+              │   Memory Engine │
+              └────────┬────────┘
+          ┌────────────┼────────────┐
+          ▼            ▼            ▼
+        local      SQLite         cloud
+        memory     durable        sync
+```
+
+## 1. The rule: the data is born local
+
+```ts
+memorio.memory.remember('user.language', 'Italian', { scope: 'local' })
+//        ↓ local first
+//   store / sessionStorage / IndexedDB / sql.js
+//        ↓ sync / push (when online)
+//   cloud provider
+```
+
+The cloud therefore does not **replace** memory: it **replicates** it. This
+gives you: offline-first, lowest latency, data available immediately,
+synchronization when online, multi-device, multi-user, centralized persistence.
+
+We deliberately do **not** provide:
+
+```ts
+// ❌ two mental models
+memory.cloud.save(...)
+```
+
+Instead:
+
+```ts
+memorio.memory.remember('user.language', 'Italian')
+// and a single configuration point:
+memorio.memory.configure({ sync: { provider: myCloudProvider, namespace: '…' } })
+```
+
+## 2. Scopes (isolation, not a security boundary)
+
+| Scope | Lifetime | Syncs by default |
+|---|---|---|
+| `'device'` | this browser/device only | no (sticky) |
+| `'user'` | follows the user across devices | yes (requires provider + namespace) |
+| `'shared'` | shared across users / tenant | yes (requires provider + namespace) |
+
+> As with `memorio.createContext`, **scoping is a naming convention, not a
+> security boundary.** Enforce real isolation server-side.
+
+## 3. SQLite as the local durable store
+
+SQLite (`sql.js`) is **in-memory by default** (volatie per page load). It becomes
+the durable journal/value store when you opt in:
+
+- `sqlite.config({ persistence: true })` / `sqlite.db.create('app', { persistence: true })`
+  snapshot the database to `store` (localStorage) and restore it on reopen.
+- Writes are snapshotted via sql.js `updateHook` (debounced).
+- `sqlite.db.persist(name)` forces an immediate save; `sqlite.db.close(name)`
+  flushes + closes; `sqlite.db.download(name, file?)` triggers a browser
+  `.sqlite` download (dev convenience).
+
+See `docs/markdown/SQLITE.md` for the full SQLite reference.
+
+## 4. The local operation journal
+
+The **sync journal** is the durable op log that drives cloud reconciliation.
+It is persisted on `store` (localStorage) - **not** on an in-memory sql.js db,
+because pending operations must survive a refresh for offline-first to work.
+
+| Method | Returns | Notes |
+|---|---|---|
+| `memory.journal.append(entry, operation)` | `Promise<MemoryEntry>` | records `remember\|update\|forget\|expire\|confirm\|supersede` with `sync:'pending'` |
+| `memory.journal.pending()` | `Promise<MemoryEntry[]>` | rows where `sync != 'synced'`, for the current namespace |
+| `memory.journal.markSynced(ids)` | `Promise<number>` | advances rows to `synced` (namespace-scoped) |
+| `memory.journal.get(id)` | `Promise<MemoryEntry \| null>` | single entry, namespace-scoped |
+| `memory.journal.clear()` | `Promise<void>` | wipes the current namespace's journal |
+| `memory.journal.replay()` | `Promise<SyncAck>` | pushes `pending()` to the provider, marks synced, optional `pull` |
+| `memory.journal.status()` | `Promise<'store'>` | the substrate in use |
+
+We sync **operations of memory**, never a raw database dump:
+
+```
+user A        device A
+remember X ─────► local ─────► sync ─────► cloud
+forget Z ──────► local ─────► sync ─────► cloud
+```
+
+## 5. Conflict resolution
+
+The cloud must not simply say "last write wins." Memorio tags every entry with:
+
+- `confidence` (0–1, user/system trust in the value)
+- `lastConfirmedAt` / `updatedAt` (epoch ms)
+- `version` (monotonic per-key counter)
+- `source` / `scope`
+
+Remote conflicts are surfaced as `sync:'conflict'` rows via
+`journal.pending()`; the provider's `resolve(op)` hint decides locally. Example:
+
+```
+Laptop:  language=Italian, confidence=0.92
+Phone:   language=English, confidence=0.61
+→ higher-confidence entry wins locally; the provider decides for shared scope.
+```
+
+## 6. Configuring a backend
+
+Sync is **opt-in**. You supply an application-owned `provider` that knows how to
+talk to your backend (REST, WebSocket, Supabase, a custom agent server, …).
+
+```ts
+memorio.memory.configure({
+  namespace: 'user:123:device:abc',   // tenant/user/device - partitions the journal
+  provider: {
+    push(ops) { return fetch('/api/sync', { method: 'POST', body: JSON.stringify(ops), headers: authHeaders }) }
+    pull(since) { return fetch(`/api/sync?since=${since}`).then(r => r.json()) }
+    resolve(op) { return op.confidence >= 0.8 ? 'local' : 'remote' }
+  },
+  auto: true                           // auto-replay on focus/online (default true)
+})
+```
+
+```ts
+interface SyncProvider {
+  push(ops: MemoryEntry[]): Promise<{ synced: string[]; conflicts?: string[]; error?: string }>
+  pull?(since?: number): Promise<MemoryEntry[]>
+  resolve?(op: MemoryEntry): Promise<'local' | 'remote' | 'merge'>
+}
+```
+
+`memorio.memory.ready` resolves once the local journal substrate is chosen.
+
+## 7. Security (NIST / OWASP / NSA posture)
+
+- **Memorio never handles credentials.** No passwords, tokens, or API keys are
+  read from or stored by memorio. Authentication/authorization live in your
+  `provider`/backend (OWASP A01: Broken Access Control).
+- **Namespace isolation.** The journal is keyed by `namespace:id` at the storage
+  layer; there is **no API** to enumerate or open another namespace's journal. A
+  client holding a forged/fake namespace simply sees its own (empty) journal.
+- **No dynamic code.** Journal entries are strictly JSON-round-tripped,
+  size-capped (10 MB/entry), and never `eval`'d. The sql.js loader never
+  `import()`s a bare specifier that could be hijacked at build time.
+- **Trust boundary:** memorio owns the local durable copy + operation log; the
+  provider/backend owns remote-side auth and conflict resolution. Memorio
+  surfaces `conflict`/`error` rows; it does not fabricate a winner.
+- **Data-at-rest (NSA/CISA).** memorio's `store`/`idb`/`sqlite` snapshots are
+  **not encrypted**. If you persist user data server-side or ship it through your
+  backend, encrypt it server-side with keys you manage - memorio treats the local
+  store as untrusted-from-the-browser and does not attest its own integrity.
+
+## 8. Where data lives
+
+| Substrate | API | Volatile? | Persistent? |
+|---|---|---|---|
+| in-memory `Proxy` | `state` | yes (per tab) | no |
+| `localStorage` / Map | `store` | no | yes (browser) |
+| `sessionStorage` / Map | `session` | no | per-tab (browser) |
+| IndexedDB | `idb`, `memory` durable | no | yes |
+| sql.js (WASM heap) | `sqlite` | **yes** | only with `persistence: true` (snapshot → `store`) |
+| sync journal | `memory.journal` | no | yes (`store`) |
+
+---
+
+## 9. Evolving the journal toward multi-device consistency
+
+Moving from sequential, single-device sync to concurrent offline edits across
+multiple devices is a classic local-first challenge. Below are six
+architectural strategies, in increasing order of sophistication, that can be
+layered onto the existing journal **without** adopting a full CRDT framework
+(Yjs, Automerge, etc.).
+
+> **Scope note.** These strategies target `memorio.memory` first - it already
+> carries the metadata a journal needs (`confidence`, `source`, `tag`, `scope`,
+> `createdAt`, `lastConfirmedAt`). If synchronization is ever extended to
+> `state` or `store`, those layers must gain HLC timestamps and path-level
+> fields explicitly - they cannot inherit them from `memory`.
+
+### 9.1 Field-level / path-level journaling
+
+Recording an entire entity in the journal makes orthogonal edits un-mergeable:
+
+```json
+{ "op": "set", "path": "user.role", "value": "admin", "timestamp": 1710000000 }
+```
+
+Device A writes `user.name`, device B writes `user.role` - both are patch
+operations on the **same entity**. An entity-level journal would produce one
+opaque `UPDATE user = {…}` and one write would clobber the other. Path-level
+patches merge automatically because the paths are disjoint.
+
+> **Not a panacea.** Path-level journaling merges edits to *different* fields.
+> Two devices writing the **same** path concurrently still need explicit conflict
+> resolution (Section 5). HLC tells you *when* the events happened; it does not
+> tell you *which value wins* when events are truly concurrent on the same path.
+
+### 9.2 Causal ordering with Hybrid Logical Clocks (HLC)
+
+Wall-clock timestamps alone fail under clock drift. Associate every journal
+entry with an HLC that combines a physical component, a logical counter, and a
+node identifier:
+
+```
+hlc:1710000005:2:deviceB
+```
+
+This gives constant-size causal ordering (vs. vector clocks, which grow with
+the number of writers - problematic for a bounded journal). The sync engine can
+then apply causally-dependent operations in order and invoke the conflict
+resolver only for genuinely concurrent writes on the same path.
+
+### 9.3 Ordering collections without full OT - fractional indexing
+
+Arrays and ordered lists are the hardest non-CRDT case. Numeric indices shift
+when a peer inserts or deletes nearby. Two lightweight options:
+
+1. **Keyed collections** - treat list items as a `Map<id, value>` rather than a
+   positional array. No index renumbering needed.
+2. **Fractional indexing** - assign each element a sortable key between its
+   neighbours (e.g. `1.0`, `2.0` → insert at `1.5`). On repeated re-inserts
+   between the same pair, keys grow in length and should be rebalanced
+   periodically. Use a mature library (`fractional-indexing` on npm) rather than
+   reimplementing the arithmetic.
+
+### 9.4 Explicit deletions (tombstones)
+
+A bare "remove" entry can be resurrected as a "zombie" when a concurrent
+update is replayed after it. Instead, record every `forget` / `delete` as a
+first-class journal event with its own HLC:
+
+```json
+{ "op": "delete", "path": "user.role", "timestamp": "hlc:1710000005:0:deviceA" }
+```
+
+The tombstone participates in the same causal comparison as `set` events: if
+the delete's HLC succeeds the update's, the field is gone; if it precedes, the
+update is re-applied. Keep tombstones around until all known peers have
+acknowledged them, then garbage-collect during a maintenance sweep.
+
+### 9.5 Hybrid sync - server-assisted consensus
+
+Since memorio's cloud role is transport-and-acknowledgement (not source of
+truth), the backend can resolve conflicts on the server and return the
+canonical sequence:
+
+| Pattern | Description |
+|---|---|
+| **Optimistic local apply** | Apply the local journal entry immediately and emit reactive events. |
+| **Server ack** | Backend validates causal order against the central state and returns the official sequence. |
+| **Client journal rebase** | Confirmed entries are purged from the local journal; unconfirmed local entries are replayed on top of the acknowledged state. |
+
+### 9.6 Validation - convergence simulation
+
+Causal correctness is theoretical until you test it across replay orderings:
+
+- Generate random concurrent `set` / `delete` operations across N simulated
+  devices (shared paths and disjoint paths).
+- Replay the operation log in every plausible ordering on each simulated
+  device.
+- Assert **state convergence**: every device arrives at the same final state
+  regardless of delivery order.
+- Include hand-crafted pathological cases (`update` vs concurrent `delete` on
+  the same path, `insert` vs concurrent `delete` on the same array index,
+  interleaved reorders).
+
+### 9.7 Migration
+
+Path-level journal entries with HLC and tombstones are a format change from
+entity-level entries. Mitigate with:
+
+- An explicit **version header** on every journal entry.
+- A clear migration policy: either a one-time compaction pass that folds
+  legacy entity-level entries into the current state and starts a fresh
+  journal, or a dual-format reader that can replay both formats during the
+  transition window.
+
+### 9.8 Recommended architecture diagram
+
+```text
+[ Local Mutation ]
+        │
+        ▼
+[ Field-Level Patch (set/delete) + HLC Timestamp ]
+        │
+        ├───► Local State (immediate reactive update)
+        │
+        └───► Local Journal (incl. tombstones for deletes)
+                    │
+             (Online Sync)
+                    │
+                    ▼
+        [ Backend Conflict Resolver ]
+        (concurrent path → resolveConflict;
+         otherwise apply in HLC order)
+                    │
+                    ▼
+        [ State Ack / Rebased Journal ]
+```
+
+Applying granular path-level patches, HLC for causality, fractional indexing
+for ordered collections, explicit tombstones for deletions, and a convergence
+test suite lets memorio handle high-frequency concurrent offline edits across
+devices without the overhead of a full CRDT stack.