memorio 4.7.1 → 4.8.0

package/markdown/MEMORY-ATTACHMENT.md

@@ -0,0 +1,95 @@
+# Node Attachment System
+
+`memorio.memory` extends with a **Node Attachment System** allowing memory elements to dynamically attach, reference, or connect to other memory elements at runtime.
+
+This is an **extension**, not a replacement, of existing memory semantics.
+
+## Concepts
+
+```text
+Memory
+ └── Node
+      ├── identity
+      ├── value
+      ├── metadata
+      └── relationships
+            └── Edge
+                 ├── target
+                 ├── relation
+                 ├── metadata
+                 ├── confidence
+                 └── lifecycle
+```
+
+## API
+
+```ts
+const user = await memory.remember("user", { name: "Alice" })
+const project = await memory.remember("project", { name: "Memorio" })
+
+await user.connect(project, { relation: "works-on", confidence: 0.95 })
+await user.attach(project)  // → connect with relation "attached-to"
+
+const related = await user.related({ direction: "both", relation: "works-on" })
+const tree = await user.traverse({ depth: 3 })
+```
+
+### Methods
+
+| Method | Description |
+|--------|-------------|
+| `node.connect(target, opts?)` | Create a directed relationship |
+| `node.disconnect(target, opts?)` | Remove a relationship |
+| `node.related(opts?)` | Query related nodes |
+| `node.traverse(opts?)` | Bounded graph traversal |
+| `memory.connect(source, target, opts?)` | Low-level connect |
+| `memory.disconnect(source, target, opts?)` | Low-level disconnect |
+
+## Scopes
+
+Relationship visibility respects memory scopes — a query in one scope cannot leak inaccessible nodes from another.
+
+## Persistence
+
+Relationships use **IDs**, never embedded objects:
+
+```json
+{
+  "id": "user",
+  "relationships": [
+    { "target": "project", "relation": "works-on" }
+  ]
+}
+```
+
+## Lifecycle
+
+- **TTL**: independent per node/edge — does not propagate
+- **Confidence**: independent — node and edge have separate confidence values
+- **Deletion**: deleting a node removes its edges (default policy)
+- **Cycles**: supported with visited-set protection during traversal
+- **Dangling references**: `retain` by default (configurable)
+
+## Security
+
+Relationship traversal inherits the same scope/access rules as direct retrieval. A user cannot gain access to Node B merely because Node A connects to it.
+
+## Experimental Substrate Evaluation
+
+Runtime substrate choices (Object Graph, WeakMap, DOM) are under benchmark evaluation. See:
+
+- [Leak Detection Protocol](../experimental/memory-substrates/LEAK-DETECTION.md)
+
+## Serialization
+
+Nodes serialize to IDs only — never recursively embedded graphs.
+
+## Performance
+
+- `O(1)` indexed lookup for direct node retrieval
+- Graph traversal only on explicit request
+- Indexable by `source`, `target`, `relation`
+
+## Backward Compatibility
+
+Existing APIs (`remember`, `get`, `forget`, `search`) continue to work unchanged. A node-enabled memory entry remains usable as a normal memory entry.

package/markdown/MEMORY.md

@@ -0,0 +1,155 @@
+# Memory System
+
+`memorio.memory` provides a semantic memory layer — a key/value store with type safety, TTL, confidence scoring, tagging, and scope-based persistence.
+
+## Core API
+
+### `memorio.memory.remember(key, value, opts?)`
+
+Stores a memory entry.
+
+```ts
+await memorio.memory.remember('user.name', 'Dario', {
+  type: 'preference',
+  confidence: 0.92,
+  scope: 'local',
+  ttl: null,
+  tags: ['user', 'profile'],
+  source: 'conversation'
+})
+```
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `id` | `string` | `key` | Custom entry ID |
+| `type` | `'fact' \| 'preference' \| 'decision' \| 'task' \| 'context'` | `'fact'` | Semantic classification |
+| `confidence` | `number` | `1.0` | Trust score 0.0–1.0 |
+| `scope` | `'hot' \| 'session' \| 'local' \| 'durable'` | auto | Storage backend (see below) |
+| `ttl` | `number \| null` | `null` | Milliseconds before expiry (`null` = never) |
+| `tags` | `string \| string[]` | `[]` | Metadata for filtering |
+| `source` | `string` | `undefined` | Origin of the memory |
+
+### `memorio.memory.recall(query, opts?)`
+
+Retrieves a memory by key.
+
+```ts
+const name = await memorio.memory.recall('user.name')
+const highConfidence = await memorio.memory.recall('project', {
+  minConfidence: 0.8
+})
+```
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `type` | `MemoryType \| MemoryType[]` | `*` | Filter by type |
+| `tags` | `string \| string[]` | `*` | Filter by tags |
+| `minConfidence` | `number` | `0` | Minimum confidence threshold |
+| `includeObsolete` | `boolean` | `false` | Include expired/superseded entries |
+
+### `memorio.memory.update(key, value, opts?)`
+
+Updates an existing memory. Creates a **superseded copy** of the old entry (preserving history).
+
+```ts
+await memorio.memory.update('user.name', 'Alice', { confidence: 0.95 })
+```
+
+### `memorio.memory.forget(key)`
+
+Permanently removes a memory.
+
+```ts
+await memorio.memory.forget('user.temp')
+```
+
+### `memorio.memory.context(opts?)`
+
+Returns ranked, relevant memories. Uses recency × confidence × type/scope boost algorithm.
+
+```ts
+const ctx = await memorio.memory.context({
+  tags: 'project',
+  types: ['decision', 'preference'],
+  minConfidence: 0.7,
+  maxEntries: 10,
+  scopes: ['local', 'durable']
+})
+```
+
+Returns:
+```ts
+Array<{
+  key: string
+  value: any
+  type: MemoryType
+  confidence: number
+  scope: MemoryScope
+  tags: string[]
+  age: number          // milliseconds since createdAt
+  accessCount: number
+  lastAccessedAt: number
+}>
+```
+
+### `memorio.memory.stats()`
+
+Returns usage statistics:
+
+```ts
+{
+  total: number
+  byScope: Record<MemoryScope, number>
+  byType: Partial<Record<MemoryType, number>>
+  expired: number
+}
+```
+
+### `memorio.memory.forgetExpired()`
+
+Cleans all expired entries. Returns count removed.
+
+### `memorio.memory.clear()`
+
+Wipes all memories and the index.
+
+## Memory Entry Model
+
+```ts
+interface MemoryEntry<T = any> {
+  id: string
+  key: string
+  value: T
+  type: 'fact' | 'preference' | 'decision' | 'task' | 'context'
+  confidence: number
+  scope: 'hot' | 'session' | 'local' | 'durable'
+  ttl?: number | null
+  tags: string[]
+  source?: string
+  status: 'active' | 'obsolete' | 'superseded'
+  createdAt: number
+  lastConfirmedAt: number
+  supersededId?: string | null
+}
+```
+
+## Scopes
+
+| Scope | Storage | TTL | Cross-session | Size limit |
+|-------|---------|-----|---------------|------------|
+| `hot` | `state` proxy | ✅ | ❌ | ~5MB (RAM) |
+| `session` | `sessionStorage` | ✅ | Tab only | ~5MB |
+| `local` | `localStorage` | ✅ | ✅ | ~10MB |
+| `durable` | `IndexedDB` | ✅ | ✅ | ~1GB+ |
+
+Default scope is `local` for values ≤100KB, `durable` for larger values.
+
+## Memory Lifecycle
+
+| Event | Behavior |
+|-------|----------|
+| `remember()` on existing key | Old entry → `superseded`, new entry → `active` |
+| Entry with TTL expires | Status → `obsolete` (cleaned by `forgetExpired()`) |
+| `recall()` expired entry | Returns `null` unless `includeObsolete: true` |
+| `update()` | Creates superseded copy + updated active entry |
+| `clear()` | Wipes all memories + index |

package/markdown/SCHEMA.md

@@ -0,0 +1,169 @@
+# Schema Validation - Memorio
+
+> ✅ **Universal**: Works in Browser, Node.js, Deno, and Edge Workers
+
+Schema validation guards your `state` against invalid writes. It runs inside the state proxy's `set` trap, so any `state.somePath = value` that violates a registered schema is rejected at runtime — before the value is ever stored.
+
+Schema validation is **opt-in** and **zero-dependency**.
+
+---
+
+## Quick Start
+
+```javascript
+import 'memorio'
+
+// Register a validator for a top-level state key
+memorio.registerSchema('user', {
+  type: 'object',
+  required: ['name', 'email'],
+  properties: {
+    name: { type: 'string', min: 1 },
+    email: { type: 'string', pattern: /^[^@]+@[^@]+$/ },
+    age: { type: 'number', min: 0, max: 150 }
+  }
+})
+
+// Valid write — accepted
+state.user = { name: 'Sara', email: 'sara@test.com', age: 30 }
+
+// Invalid write — rejected, returns false
+state.user = { name: 'Sara' }           // missing 'email'
+state.user = { name: 42, email: 'x' }   // wrong type for 'name'
+state.user = { age: -5 }                // out of range
+```
+
+---
+
+## Schema Definition
+
+A `Schema` object supports the following fields:
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `type` | `'string' \| 'number' \| 'boolean' \| 'object' \| 'array' \| 'any'` | Runtime type check |
+| `required` | `string[]` | Property names that must exist (objects only) |
+| `properties` | `Record<string, Schema>` | Nested property schemas (validated recursively) |
+| `min` | `number` | Number: minimum value. String: minimum length |
+| `max` | `number` | Number: maximum value. String: maximum length |
+| `pattern` | `RegExp` | Regex the string value must match |
+| `enum` | `any[]` | Whitelist of allowed values |
+| `validator` | `(value) => boolean \| string` | Custom validator function |
+
+### Custom validator functions
+
+For logic that's hard to express declaratively, pass a function instead of a schema object:
+
+```javascript
+memorio.registerSchema('counter', (value) => {
+  if (typeof value !== 'number') return 'counter must be a number'
+  if (value < 0) return 'counter must be >= 0'
+  return true
+})
+```
+
+A custom validator receives the raw value. Return `true` to accept, or a **string** describing the error to reject.
+
+---
+
+## Path-based registration
+
+Schemas are keyed by their **state path**, relative to `state`:
+
+| API call | Catches |
+|----------|---------|
+| `registerSchema('user', schema)` | `state.user = value` |
+| `registerSchema('user.age', schema)` | `state.user.age = value` |
+| `registerSchema('items', schema)` | `state.items = value` |
+
+The full dotted path is constructed from the proxy's tree depth. Nested sets propagate the full path automatically.
+
+---
+
+## Manual validation
+
+You can validate a value without writing it to state:
+
+```javascript
+memorio.validate('user', { name: 'Sara', email: 'sara@test.com' })
+// { valid: true }
+
+memorio.validate('user', { name: 'Sara' })
+// { valid: false, errors: ["user: missing required property 'email'"] }
+```
+
+When no schema is registered for a path, `validate` returns `{ valid: true }`.
+
+---
+
+## Schema management
+
+```javascript
+memorio.listSchemas()           // ['user', 'theme', 'items', 'counter']
+memorio.unregisterSchema('counter')  // removes the schema
+```
+
+---
+
+## Full API
+
+| Method | Parameters | Returns | Description |
+|--------|-----------|---------|-------------|
+| `memorio.registerSchema(path, schema)` | `string`, `Schema \| fn` | `void` | Register a validator |
+| `memorio.validate(path, value)` | `string`, `any` | `{ valid, errors? }` | Manually validate a value |
+| `memorio.unregisterSchema(path)` | `string` | `boolean` | Remove a registered schema |
+| `memorio.listSchemas()` | none | `string[]` | List all registered paths |
+| `memorio.registerSchema()` is also importable | `registerSchema` | named export | same function |
+
+---
+
+## Combine with Typed Stores
+
+Schema validation gives you **runtime** safety; typed stores give you **compile-time** safety. Use both for full coverage:
+
+```typescript
+import 'memorio'
+
+interface AppState {
+  user: { name: string; email: string; age: number }
+  theme: 'light' | 'dark'
+}
+
+const app = memorio.typed<AppState>()
+
+memorio.registerSchema('user', {
+  type: 'object',
+  required: ['name', 'email'],
+  properties: {
+    name: { type: 'string', min: 1 },
+    email: { type: 'string', pattern: /^[^@]+@[^@]+$/ },
+    age: { type: 'number', min: 0, max: 150 }
+  }
+})
+
+app.user = { name: '', email: 'bad' }   // ❌ TypeScript: age missing
+                                         // ❌ Runtime: missing required fields
+app.user = { name: 'Sara', email: 'ok', age: 30 } // ✅ both checks pass
+```
+
+See [Typed Stores](TYPED.md) for compile-time type safety.
+
+---
+
+## How it works
+
+1. When you call `registerSchema(path, schema)`, the schema is stored in an internal `Map`.
+2. On every `state.set` operation, the proxy's `set` trap computes the full path (e.g. `'user.name'`).
+3. If a schema is registered for that path, the value is validated.
+4. If validation fails, the write is rejected (`return false`), and an error is logged via `console.error` (when `memorio.debug = true`) or `console.debug` (via the internal `message` helper).
+5. If no schema is registered, the write proceeds normally.
+
+The validation adds negligible overhead when no schemas are registered (a single `Map` lookup that returns `undefined`).
+
+---
+
+## Limitations
+
+- Schema validation hooks into the `state` proxy only. `store`, `session`, and `cache` are not validated (they use separate storage). Use `validate()` before writing to other modules.
+- Path matching is **exact**: `registerSchema('user')` guards `state.user = ...`, but does **not** recursively validate `state.user.name = 'new'`. Register schemas at each path you need to guard.
+- The schema system is not a replacement for server-side validation. It protects against accidental misuse and provides defense-in-depth in the browser.

package/markdown/SECURITY.md

@@ -310,7 +310,7 @@ If you discover a security vulnerability in Memorio:
 
 ---
 
-### v2.7.0 — Previous
+### v2.7.0 - Previous
 
 - ✅ Added `crypto.getRandomValues()` fallback
 - ✅ Added key length validation (512 chars)

package/markdown/SQLITE.md

@@ -0,0 +1,181 @@
+# SQLite - Memorio
+
+> The `sql.js` package is **optional** — if it is not installed, Memorio loads `sql.js` on first use by injecting the `sql-wasm-browser.js` UMD build from the jsDelivr CDN as a classic `<script>` (which exposes the `initSqlJs` factory on `globalThis`). No bare `import('sql.js')` is ever emitted, so bundlers never resolve the optional dependency at build time. Not available in Node.js/Deno.
+
+SQLite brings a full SQL engine into the browser through [`sql.js`](https://sql.js.org), a port of SQLite to JavaScript via WebAssembly. Memorio loads it **lazily** on first use, so it only affects bundles that actually use the `sqlite` module.
+
+## Installation
+
+```bash
+npm install memorio
+# Optional: install the SQLite engine locally (the default loader fetches it from the CDN)
+npm install sql.js
+```
+
+```javascript
+import 'memorio';
+```
+
+> The default loader injects the sql.js UMD build (`sql-wasm-browser.js`) from the jsDelivr CDN as a classic `<script>`, exposing the `initSqlJs` factory on `globalThis`. This keeps `sql.js` (and its WASM) out of your build graph entirely — it's only fetched when `sqlite` is first used. For production/self-hosted setups, install `sql.js` and configure a custom `loader` (e.g. `sqlite.config({ loader: () => import('sql.js') })` under a bundler), or set the wasm base with `sqlite.config({ wasmUrl })` / `sqlite.config({ locateFile })`.
+
+---
+
+## Quick Examples
+
+### Example 1: Basic Usage
+
+```javascript
+// Create an in-memory database and a table
+await sqlite.db.create('myApp');
+await sqlite.query.run('myApp', 'CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)');
+
+// Insert data
+await sqlite.query.run('myApp', 'INSERT INTO users (name) VALUES (?)', ['Mario']);
+
+// Select data
+const rows = await sqlite.query.select('myApp', 'SELECT * FROM users');
+console.debug(rows); // [{ id: 1, name: 'Mario' }]
+```
+
+### Example 2: CRUD shortcuts
+
+```javascript
+const db = 'store';
+
+await sqlite.db.create(db);
+await sqlite.query.run(db, 'CREATE TABLE products (id INTEGER PRIMARY KEY, name TEXT, price REAL)');
+await sqlite.data.set(db, 'INSERT INTO products (name, price) VALUES (?, ?)', ['Apple', 1.5]);
+await sqlite.data.set(db, 'INSERT INTO products (name, price) VALUES (?, ?)', ['Banana', 0.8]);
+
+// First matching row
+const apple = await sqlite.data.get(db, 'SELECT * FROM products WHERE name = ?', ['Apple']);
+console.debug(apple); // { id: 1, name: 'Apple', price: 1.5 }
+```
+
+### Example 3: Export / Import
+
+```javascript
+await sqlite.db.create('myApp');
+await sqlite.query.run('myApp', 'CREATE TABLE todos (id INTEGER PRIMARY KEY, task TEXT)');
+await sqlite.data.set('myApp', 'INSERT INTO todos (task) VALUES (?)', ['Write docs']);
+
+// Export the in-memory database to a portable binary dump
+const dump = await sqlite.db.export('myApp');
+
+// Later... import it back
+await sqlite.db.import('restored', dump);
+const rows = await sqlite.query.select('restored', 'SELECT * FROM todos');
+```
+
+### Example 4: Engine control
+
+```javascript
+// Configure the wasm location BEFORE first use (sets the locateFile base)
+sqlite.config({ wasmUrl: '/static/sql.js/' });
+
+// Or provide a fully custom loader (e.g. a local/npm build of sql.js)
+sqlite.config({ loader: async () => {
+  return await import('sql.js');   // local install — resolves at runtime
+}});
+
+// Enable automatic persistence of one or more databases (see Example 5)
+sqlite.config({ persistence: true });
+
+// Wait until the engine is ready
+await sqlite.ready;
+console.debug('SQLite version:', sqlite.db.version());
+```
+
+### Example 5: Persistence & dev download
+
+> **Important:** sql.js databases live in **WebAssembly memory** — they are
+> in-memory and **volatile** (lost on page refresh) unless you persist them.
+
+```javascript
+// Opt into auto-persistence at create time (global config or per-create)
+await sqlite.db.create('app', { persistence: true });
+
+await sqlite.query.run('app', 'CREATE TABLE notes (id INTEGER PRIMARY KEY, body TEXT)');
+// writes are snapshotted to localStorage automatically (debounced via updateHook)
+
+await sqlite.db.persist('app');          // force an immediate snapshot
+// After a refresh, reopening restores the data:
+await sqlite.db.create('app', { persistence: true });
+
+// Dev convenience: download the current db as a .sqlite file from the browser
+await sqlite.db.download('app', 'app.sqlite');
+
+// Flush + persist + release the handle
+await sqlite.db.close('app');
+```
+
+Persistence is stored on `store` (localStorage, namespaced per
+`sqlite.config({ namespace })` / memorio context) as a base64 snapshot under
+`memorio:sqlite:db:<ns>:<name>`. It is best-effort: if `store` is unavailable
+the database simply behaves as in-memory.
+
+---
+
+## API Reference
+
+### Engine Control
+
+| Method | Parameters | Returns | Description |
+|--------|------------|---------|-------------|
+| `sqlite.config(opts)` | `opts: { loader?, locateFile?, wasmUrl?, persistence?, namespace? }` | `sqlite` | Configure the engine. `loader` fully replaces the initializer (default = CDN `<script>`); `locateFile` customizes wasm resolution; `wasmUrl` sets the wasm base directory; `persistence` toggles automatic db snapshots; `namespace` partitions persisted snapshots. Chainable. |
+| `sqlite.db.create(name, opts)` | `name: string, opts?: { data?, persistence? }` | `Database` | Opens/creates a named in-memory db. With `persistence: true` (or global config), a snapshot is restored if present and writes are auto-saved. |
+| `sqlite.db.persist(name)` | `name: string` | `Promise<boolean>` | Force an immediate snapshot of a persisted database to `store`. |
+| `sqlite.db.download(name, filename?)` | `name: string, filename?: string` | `Promise<boolean>` | Dev-only: trigger a browser download of the db as a `.sqlite` file. |
+| `sqlite.db.close(name)` | `name: string` | `Promise<void>` | Persist (if enabled), close, and release the handle. |
+| `sqlite.ready` | none | `Promise<void>` | Resolves once `sql.js` has been initialized. Rejects (and sets `_disabled`) if the engine can't load. |
+| `sqlite.db.version()` | none | `string \| null` | Engine version, available after initialization. |
+| `sqlite._disabled` | none | `boolean` | `true` when the module is disabled (non-browser env / load failure). |
+| `sqlite._warning` | none | `string \| undefined` | Reason text when disabled, if applicable. |
+
+### Database Methods
+
+| Method | Parameters | Returns | Description |
+|--------|------------|---------|-------------|
+| `sqlite.db.support()` | none | `boolean` | Check whether SQLite can run in this environment. |
+| `sqlite.db.create(name, opts?)` | `name: string`, `opts?: { data }` | `Promise<Database>` | Create an in-memory database (or reopen one from a binary dump). |
+| `sqlite.db.get(name)` | `name: string` | `Database` | Retrieve an open database handle. |
+| `sqlite.db.delete(name)` | `name: string` | `boolean` | Close and remove a database handle. |
+| `sqlite.db.list()` | none | `string[]` | List open database names. |
+| `sqlite.db.size(name?)` | `name?: string` | `Promise<number>` | Size in bytes of one or all databases. |
+| `sqlite.db.export(name)` | `name: string` | `Promise<Uint8Array>` | Export a database to a binary dump. |
+| `sqlite.db.import(name, data)` | `name: string`, `data` | `Promise<Database>` | Create/replace a database from a binary dump. |
+
+### Query Methods
+
+| Method | Parameters | Returns | Description |
+|--------|------------|---------|-------------|
+| `sqlite.query.run(name, sql, params?)` | `name: string`, `sql: string`, `params?: any[]` | `Promise<number>` | Run a statement that does not return rows (INSERT / UPDATE / DELETE / CREATE). Returns modified-row count. |
+| `sqlite.query.select(name, sql, params?)` | `name: string`, `sql: string`, `params?: any[]` | `Promise<object[]>` | Run a query and return the rows as plain objects. |
+
+### Data Methods (CRUD shortcuts)
+
+| Method | Parameters | Returns | Description |
+|--------|------------|---------|-------------|
+| `sqlite.data.set(name, sql, params?)` | `name: string`, `sql: string`, `params?: any[]` | `Promise<number>` | Alias of `sqlite.query.run`. |
+| `sqlite.data.get(name, sql, params?)` | `name: string`, `sql: string`, `params?: any[]` | `Promise<object \| null>` | Alias of `sqlite.query.select` returning the first row (or `null`). |
+
+---
+
+## Platform Support
+
+| Platform | Support | Notes |
+|----------|---------|-------|
+| Browser | ✅ Full | Requires WebAssembly + the `sql.js` package (or CDN fallback) |
+| Edge Worker | ⚠️ Limited | WebAssembly may be available; configure a `loader` manually |
+| Node.js | ❌ Not available | Use `store` or `session` instead |
+| Deno | ❌ Not available | Use `store` or `session` instead |
+
+---
+
+## Best Practices
+
+1. Configure the wasm path for production: `sqlite.config({ wasmUrl })` or `sqlite.config({ loader })`.
+2. Create one named database per feature and reuse the handle: `const db = await sqlite.db.create('app')`.
+3. Always release databases you no longer need: `sqlite.db.delete('temp')`.
+4. Use `await sqlite.ready` before running statements to ensure the engine is loaded.
+5. Use `?` placeholders and `params` to avoid SQL injection.