memorio 5.0.0 → 5.1.0

package/README.md

@@ -1,8 +1,6 @@
 # 🧠 memorio
 
-**Application state intelligence runtime.**
-
-Use it like an object.
+**The memory layer for AI agents and apps — owned by the user, not the vendor.**
 
 ```ts
 import 'memorio/global'
@@ -19,18 +17,20 @@ Just data, available where your application needs it.
 
 ---
 
-## Why memorio?
-
-Most applications eventually need more than one kind of data: reactive state, persistent values, session data, caches, structured browser storage, local SQL, application memory, history, and optional sync.
+## Why memorio
 
-Usually that means a different library - and a different mental model - for each.
+Most AI-powered apps (and most apps in general) eventually need more than "just state": reactive values, persistent data, session data, caches, structured browser storage, local SQL, application memory, history, and optional sync.
 
-**Memorio gives them one consistent runtime, without making the simple case complicated.**
+Usually that means a different library — and a different mental model — for each. Memorio gives them one consistent runtime, without making the simple case complicated:
 
 ```ts
 state.value = 42
 ```
 
+But the reason memorio exists isn't just "fewer dependencies." It's this: **the memory an AI agent builds about a user shouldn't be locked inside a vendor's server, opaque, and impossible to inspect or move.** Memorio's memory layer is structured, inspectable, editable, and — when you turn on encryption — readable only with a key the vendor doesn't hold. It runs client-side, in the browser, on the edge, or in Node.
+
+That said, this covers *structured* memory - preferences, decisions, facts with confidence and provenance - not semantic similarity search. If your agent needs "find things like this conversation," pair memorio with a dedicated vector store; see [`markdown/MEMORY.md`](./markdown/MEMORY.md).
+
 Add persistence, memory, history, or sync only when your application actually needs them. The common case stays tiny; the runtime grows with you.
 
 ---
@@ -45,110 +45,23 @@ Zero production dependencies in core. Optional integrations (`react`, `sql.js`,
 
 ---
 
-## Global or explicit
-
-```ts
-// Opt-in global access
-import 'memorio/global'
-state.user = { name: 'Sara' }
-
-// Or explicit imports
-import { state, store, memory } from 'memorio'
-```
-
-Memorio never puts APIs on `globalThis` automatically and never infers dev mode from `NODE_ENV` or your bundler - global access is always something you choose.
-
----
-
-## The layers
-
-| Layer     | Purpose                          | Persistence | Reactive |
-| --------- | --------------------------------- | :---------: | :------: |
-| `state`   | application state                 |     No      |   Yes    |
-| `cache`   | transient runtime data            |     No      |    No    |
-| `session` | tab/session data                  |    Yes      |    No    |
-| `store`   | persistent key/value data         |    Yes      |    No    |
-| `idb`     | durable structured browser data   |    Yes      |    No    |
-| `sqlite`  | relational local SQL              |  Optional   |    No    |
-| `memory`  | structured application memory     |    Yes      | Optional |
-| `journal` | local-first operation history     |    Yes      |    No    |
-
-You don't have to use all of them - start with `state`.
-
-**Which one do I need?**
-
-```text
-Does the UI need to react automatically to changes?
-  yes → state
-  no  → cache (runtime only) · session (this tab) · store (small persistent kv)
-        idb (structured browser data) · sqlite (relations/SQL) · memory (what the app should remember)
-```
-
----
+## Start here
 
-## Before / after
+Three things cover most apps - reactive state, persistence, and observation:
 
 ```ts
-// Without memorio
-const [user, setUser] = useState(null)
-useEffect(() => {
-  const saved = localStorage.getItem('user')
-  if (saved) setUser(JSON.parse(saved))
-}, [])
-useEffect(() => {
-  localStorage.setItem('user', JSON.stringify(user))
-}, [user])
-
-// With memorio
-import { state, persist } from 'memorio'
-state.user = { name: 'Sara' }
-const off = persist('state.user') // restores + keeps in sync
-```
-
----
-
-## Persistent state
-
-```ts
-import { state, persist } from 'memorio'
+import { state, persist, observer } from 'memorio'
 
 state.user = { name: 'Sara' }
-const off = persist('state.user')
-
-// later
-off()
-```
 
-Nested paths work when the intermediate objects already exist: `persist('state.user.preferences.theme')`.
-Without `localStorage` (e.g. Node), `store` falls back to memory.
+persist('state.user')
 
----
-
-## Observing changes
-
-```ts
-import { observer } from 'memorio'
-
-const off = observer('state.user', (next, previous) => {
-  console.log('user changed:', next, previous)
+observer('state.user', user => {
+  console.log(user)
 })
-
-off() // unsubscribe
 ```
 
-Prefer the returned `off()` for targeted teardown (it unsubscribes only *this* callback).
-The alternatives remove listeners by path name:
-
-```ts
-observer.remove('state.user')   // removes ALL callbacks for this path
-observer.removeAll()           // removes every observer at once
-```
-
----
-
-## Application memory
-
-State stores what the application **has**. Memory stores what it **remembers**.
+And this is what makes memorio a *memory* layer, not just a state manager - structured, inspectable facts about a user or agent, not just ephemeral UI state:
 
 ```ts
 import { memorio } from 'memorio'
@@ -156,317 +69,77 @@ import { memorio } from 'memorio'
 await memorio.memory.remember('user.language', 'Italian', {
   type: 'preference',
   confidence: 0.92,
-  scope: 'local',
-  tags: ['user', 'ui'],
   source: 'conversation'
 })
 
 const language = await memorio.memory.recall('user.language')
-```
-
-Updating an entry doesn't overwrite it - the previous entry becomes `superseded`, the new one `active`:
-
-```ts
-await memorio.memory.update('user.language', 'English', { confidence: 0.95 })
-```
-
-Retrieve relevant memory instead of loading everything:
-
-```ts
-const context = await memorio.memory.context({
-  tags: 'user',
-  types: ['preference', 'decision'],
-  minConfidence: 0.7,
-  maxEntries: 10
-})
-```
-
-> **Note:** "structured memory" here means metadata-based ranking (tags, type, confidence, recency) - not embedding similarity search. Pair `memorio.memory` with a dedicated vector store if you need that.
-
-```text
-sqlite  → what data do I have?
-memory  → what does my application remember?
-```
-
----
-
-## The other layers, briefly
-
-**`store`** - persistent key/value, backed by `localStorage` in browsers, memory elsewhere.
-```ts
-store.set('preferences', { theme: 'dark' })
-const preferences = store.get('preferences')
-```
-
-**`session`** - scoped to the current browser session (`sessionStorage`). Good for wizard steps and tab-local state.
-
-**`cache`** - volatile, in-memory, disappears when the runtime disappears.
-
-**`idb`** - durable structured browser data:
-```ts
-await idb.db.create('app')
-await idb.table.create('app', 'users')
-await idb.data.set('app', 'users', { id: 1, name: 'Sara' })
-```
-
-**`sqlite`** - relational local SQL, in-memory by default via `sql.js`:
-```ts
-await sqlite.ready
-await sqlite.db.create('app', { persistence: true })
-await sqlite.query.run('app', `CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT, role TEXT)`)
-await sqlite.query.select('app', `SELECT * FROM users WHERE role = ?`, ['admin'])
-```
-> ⚠️ Persistence serializes the **entire database** on flush - fine for small/medium datasets, but large ones need deliberate batching.
-
-Check what's actually available at runtime before depending on a backend: `memorio.getCapabilities()`.
-
----
-
-## State Intelligence
-
-With history tracking enabled, mutations are recorded with a unique ID, HLC timestamp, source, and causal context - the foundation for undo/redo, diff, trace, and (planned) replay.
-
-```ts
-import { memorio } from 'memorio'
-
-memorio.enableHistory(true)
-
-state.user = { name: 'John' }
-state.user.name = 'Maria'
-state.user.name = 'Pedro'
-
-const history = memorio.trace()
-memorio.undo()
-memorio.redo()
-
-memorio.canUndo()
-memorio.canRedo()
-```
-
-**Explicit mutations & transactions:**
-
-```ts
-const mutation = memorio.mutate('state.user.role', 'admin', { source: 'permissions.enableAdmin' })
-// mutation.id · mutation.hlc · mutation.source
-
-const tx = memorio.transaction('user.migration', 'Migrate to v2 schema')
-state.user.role = 'admin'
-state.user.permissions = ['read', 'write']
-memorio.commitTransaction()
-
-// or abort and roll back
-memorio.transaction('failed-op')
-state.user.name = 'Maria'
-memorio.abortTransaction()
-```
-
-**Roadmap:**
-
-| Phase | Capability                  | Status                                             |
-| ----- | ---------------------------- | --------------------------------------------------- |
-| 1     | Mutation Engine               | ✅ Active                                            |
-| 2     | Undo / Redo / Diff / Revert   | ✅ Partial (basic trace + undo/redo shipped above)   |
-| 3     | History Engine (persisted, queryable history) | Planned                       |
-| 4     | Replay / Time Machine          | Planned                                            |
-| 5     | Causal Graph                   | Planned                                            |
-| 6     | Explain / Trace / Impact       | Planned                                            |
-| 7     | What-if Simulation             | Planned                                            |
-
----
-
-## Typed state & validation
-
-```ts
-import { memorio } from 'memorio'
-
-interface AppState {
-  user: { name: string; age: number; email: string }
-  theme: 'light' | 'dark'
-}
-
-const app = memorio.typed<AppState>()
-app.theme = 'dark'   // ✅
-app.theme = 'purple'  // ❌ TypeScript error
-
-app === state // same proxy, typed
-```
-
-Runtime schemas validate values crossing trust boundaries:
-
-```ts
-memorio.registerSchema('user', {
-  type: 'object',
-  required: ['name', 'email'],
-  properties: {
-    name: { type: 'string', min: 1 },
-    email: { type: 'string', pattern: /^[^@]+@[^@]+$/ }
-  }
-})
-
-memorio.registerSchema('tags', { type: 'array', items: { type: 'string' } })
-
-state.tags = ['a', 'b'] // accepted
-state.tags = [1, 2, 3]  // rejected
-```
-
----
-
-## React integration
-
-`useObserver` is a thin bridge over the observer engine. For the common React case,
-wrap it in a tiny helper so components read state like any other value:
-
-```tsx
-import { useReducer } from 'react'
-import { useObserver, state } from 'memorio'
-
-// Subscribe to one or more state paths and re-render on change.
-function useMemorioValue(path: any) {
-  const [, update] = useReducer((n: number) => n + 1, 0)
-  useObserver(update, path)
-  return path
-}
-
-function Counter() {
-  const counter = useMemorioValue(state.counter)
-  return <div>{counter}</div>
-}
-```
-
-`useObserver` can also auto-discover dependencies — pass a callback with no explicit
-deps and it tracks every state path read inside it — but the explicit-deps form above
-is cheaper and easier to reason about at scale.
-
-Memorio itself stays framework-independent - React is an integration, not a requirement.
-
----
-
-## Redux and other state managers
-
-> **Your state manager owns application state. Memorio owns memory.**
-
-```ts
-import { createMemorioReduxBridge } from 'memorio/redux'
-
-const memorioRedux = createMemorioReduxBridge<AppState>({
-  mappings: {
-    'user.preferences': {
-      selector: state => state.user.preferences,
-      type: 'preference',
-      scope: 'local',
-      tags: ['app', 'user']
-    },
-    'user.name': state => state.user.name
-  },
-  whitelist: ['user/preferencesChanged', 'user/nameChanged'],
-  debug: true
-})
-
-const store = createStore(reducer, applyMiddleware(memorioRedux.middleware))
-
-await memorioRedux.hydrate(store, {
-  onHydration(values) { store.dispatch(restorePreferences(values)) }
-})
-```
-
-```text
-Redux ─────────────→ memorio        memorio ───────────→ Redux
-application state    durable memory                     bootstrap hydration (one-shot)
-```
-
-The bridge persists only mapped selectors, never mirrors the full tree, coalesces bursts, and never breaks Redux dispatch if Memorio fails.
-
-| | Redux | Memorio |
-|---|---|---|
-| Core model | single store | independent runtime layers |
-| State changes | actions + reducers | direct reactive mutation |
-| Persistence | external integration | built in |
-| Structured memory | - | native |
-| Offline memory sync | - | supported |
-| Time-travel debugging | mature | State Intelligence (in progress) |
-| Production dependencies | ecosystem-dependent | zero in core |
-
-Use Redux when you need its action-pipeline discipline and ecosystem. Use Memorio when you want state, persistence, and structured memory under one runtime. You can use both.
-
----
-
-## Local-first sync
-
-Memorio synchronizes **operations** (`remember`, `update`, `forget`, `expire`, `confirm`, `supersede`), not database dumps. A local journal keeps the app working through network failures; the cloud is optional transport, not a prerequisite.
-
-```ts
-memorio.memory.configure({
-  namespace: 'user:123:device:abc',
-  provider: {
-    push: ops => fetch('/api/sync', { method: 'POST', body: JSON.stringify(ops) }),
-    pull: since => fetch(`/api/sync?since=${since}`).then(r => r.json())
-  },
-  auto: true
-})
-```
-
-Concurrent operations use HLC ordering by default; override with a custom resolver:
-
-```ts
-memorio.memory.configure({
-  resolveConflict(local, remote) {
-    if (remote.source === 'user-correction') return remote
-    return local.confidence >= remote.confidence ? local : remote
-  }
-})
-```
-
-The sync provider is still responsible for the final server-side conflict policy.
-
----
-
-## Cross-platform support
-
-| API                             | Browser    | Node.js          | Deno              | Edge / Workers        |
-| -------------------------------- | :--------: | :---------------: | :----------------: | :---------------------: |
-| `state` / `observer` / `cache`   |     ✅     |         ✅          |         ✅          |            ✅            |
-| `store` / `session`              | persistent | memory fallback   | memory fallback    | capability-dependent    |
-| `idb`                            |     ✅     |         ❌          |         ❌          | capability-dependent    |
-| `sqlite`                         |     ✅     |         ❌          |         ❌          | capability-dependent    |
-| `memory` / `journal`             |     ✅     |         ✅          |         ✅          |            ✅            |
-| `devtools`                       |  dev-only  |         ❌          |         ❌          | capability-dependent    |
-
-```ts
-memorio.getCapabilities()
-memorio.isBrowser() // .isNode() · .isDeno() · .isEdge()
-```
-
-Don't assume every persistence backend exists in every runtime - check first.
-
-> **SQLite on Node.js / Deno:** Memorio's `sqlite` is backed by `sql.js` (SQLite compiled to WebAssembly), which requires browser APIs (`window`, `document`, `WebAssembly`). It is not available in Node.js or Deno. For server-side SQL, pair Memorio with a native SQLite package directly; use `store` or `idb` for in-process persistence from Memorio.
-
----
-
-## Security
-
-Core has zero production dependencies. No `eval`, no dynamic code execution, no bundled telemetry, sanitized keys, validated inputs, bounded journal entries, UUID-based session identifiers. Independently checkable via Socket.dev / Snyk.
-
-> ⚠️ **Memorio does not encrypt data by default** - this applies to every layer (`state`, `store`, `session`, `idb`, `memory`, `sqlite`). Treat browser storage as client-controlled. Use proper encryption and backend authorization for credentials or regulated data.
-
-**Contexts are not authorization:**
-
-```ts
-memorio.createContext('tenant-123')
-```
-
-Contexts organize and isolate application concerns - they're not a security boundary. Code in the same JS runtime can potentially reach other contexts. Real tenant isolation belongs at the auth/backend layer.
-
-DevTools expose data for development only, and only when you opt into `memorio/global`.
+// { value: 'Italian', confidence: 0.92, source: 'conversation', ... }
+```
+
+For most apps, that's enough. Everything below is an optional capability you can add when your application needs it - each one documented in its own reference file.
+
+---
+
+## Where to go next
+
+### Core
+| Topic | Reference |
+| --- | --- |
+| Reactive state | [`markdown/STATE.md`](./markdown/STATE.md) |
+| Persistent key/value store | [`markdown/STORE.md`](./markdown/STORE.md) |
+| Observing changes | [`markdown/OBSERVER.md`](./markdown/OBSERVER.md) |
+| Structured application memory | [`markdown/MEMORY.md`](./markdown/MEMORY.md) |
+
+### Storage
+| Topic | Reference |
+| --- | --- |
+| Session storage (tab-scoped) | [`markdown/SESSION.md`](./markdown/SESSION.md) |
+| In-memory cache | [`markdown/CACHE.md`](./markdown/CACHE.md) |
+| IndexedDB | [`markdown/IDB.md`](./markdown/IDB.md) |
+| SQLite (`sql.js` / `bun:sqlite`) | [`markdown/SQLITE.md`](./markdown/SQLITE.md) |
+
+### Sync & history
+| Topic | Reference |
+| --- | --- |
+| Local-first sync & cloud | [`markdown/SYNC.md`](./markdown/SYNC.md) |
+| Memory attachments (linking memory entries) | [`markdown/MEMORY-ATTACHMENT.md`](./markdown/MEMORY-ATTACHMENT.md) |
+| History, undo/redo, snapshot, diff, trace | [`markdown/HISTORY.md`](./markdown/HISTORY.md) |
+
+### Integrations
+| Topic | Reference |
+| --- | --- |
+| React (`useObserver`) | [`markdown/USEOBSERVER.md`](./markdown/USEOBSERVER.md) |
+| Redux bridge | [`markdown/REDUX.md`](./markdown/REDUX.md) |
+| Typed state | [`markdown/TYPED.md`](./markdown/TYPED.md) |
+| Schema validation | [`markdown/SCHEMA.md`](./markdown/SCHEMA.md) |
+| Pub/sub events outside React | [`markdown/DISPATCH.md`](./markdown/DISPATCH.md) |
+| Console logging middleware | [`markdown/LOGGER.md`](./markdown/LOGGER.md) |
+
+### Ops & security
+| Topic | Reference |
+| --- | --- |
+| Platform detection & multi-tenant context isolation | [`markdown/PLATFORM.md`](./markdown/PLATFORM.md) |
+| Classic `import` vs `memorio/global` | [`markdown/IMPORT.md`](./markdown/IMPORT.md) |
+| Runtime introspection | [`markdown/INSPECT.md`](./markdown/INSPECT.md) |
+| Browser DevTools | [`markdown/DEVTOOLS.md`](./markdown/DEVTOOLS.md) |
+| Security, encryption, threat model | [`SECURITY.md`](./SECURITY.md) |
+| Internal self-assessment *(not a third-party audit)* | [`markdown/SELF-ASSESSMENT.md`](./markdown/SELF-ASSESSMENT.md) |
+
+### Other
+| Topic | Reference |
+| --- | --- |
+| Version history | [`CHANGELOG.md`](./CHANGELOG.md) |
+| Working examples - each file has a `Run:` comment at the top (typically `npx ts-node examples/<name>.ts`, or `npx ts-node --esm examples/<name>.tsx` for React examples) | [`examples/`](./examples/) |
+| Architecture decisions | [`adr/`](./adr/) |
 
 ---
 
 ## When to use something else
 
-- **A strict Redux-style architecture** - when you need action pipelines, extensive middleware, event-sourcing, or mature time-travel tooling. Memorio can integrate rather than replace.
+- **A strict Redux-style architecture** - when you need action pipelines, extensive middleware, event-sourcing, or mature time-travel tooling. Memorio can integrate rather than replace - see [`markdown/REDUX.md`](./markdown/REDUX.md).
 - **A server database** - for authoritative persistence, multi-user authorization, or server-side transactions.
-- **A dedicated vector database** - for true embedding similarity search. Memorio can still manage the lifecycle/metadata of what you retrieve from it.
-
-Memorio occupies a distinct space: **application state + history + causality + replay + impact analysis + simulation**. If you need to know *why* state changed, *who* changed it, or *what would happen* if it changed, Memorio is the tool - not just a state container.
+- **A dedicated secrets manager** - for high-privilege credentials, encryption key management, or regulated data at rest. Memorio's encryption is client-side and key-management is your responsibility - see [`SECURITY.md`](./SECURITY.md).
+- **A dedicated vector database** - for true embedding similarity search. Memorio manages structured memory (preferences, decisions, facts), not embeddings.
 
 ---
 
@@ -477,52 +150,10 @@ Memorio occupies a distinct space: **application state + history + causality + r
 3. **The cloud is optional** - sync extends the local app; it doesn't define it.
 4. **Use the right primitive** - relational data belongs in SQL, not a kv store; memory isn't the same as ordinary state.
 5. **Memory has meaning** - confidence, source, type, scope, TTL, tags, history, status: more than "just a value."
-6. **Global access is explicit** - opt in with `import 'memorio/global'`, or keep it explicit with named imports.
-7. **Tell the truth about boundaries** - not encrypted, not authorization, not embedding search unless paired with one.
-8. **Keep the common case tiny** - `state.value = 42` is a complete, valid program.
-
----
-
-## Quick recipes
-
-```ts
-// Global state
-import 'memorio/global'
-state.user = { name: 'Sara', role: 'admin' }
-
-// Explicit state
-import { state } from 'memorio'
-state.counter++
-
-// Persistent kv
-import { store } from 'memorio'
-store.set('preferences', { theme: 'dark' })
-
-// Persistent reactive state
-import { state, persist } from 'memorio'
-state.user = { name: 'Sara' }
-const off = persist('state.user')
-
-// Application memory
-import { memorio } from 'memorio'
-await memorio.memory.remember('user.language', 'Italian', {
-  type: 'preference', confidence: 0.92, source: 'conversation'
-})
-
-// Observe a value
-import { observer } from 'memorio'
-const off = observer('state.user', (next, previous) => console.log(next, previous))
-
-// Track mutations, undo, and trace history
-import { memorio } from 'memorio'
-memorio.enableHistory(true)
-memorio.mutate('state.user.role', 'admin', { source: 'permissions.save' })
-const tx = memorio.transaction('migration', 'v2 schema migration')
-state.user.v2 = true
-memorio.commitTransaction()
-memorio.undo()
-console.debug(memorio.trace())
-```
+6. **Memory belongs to whoever it's about** - structured and inspectable by default, encryptable with a key the vendor doesn't have to hold.
+7. **Global access is explicit** - opt in with `import 'memorio/global'`, or keep it explicit with named imports.
+8. **Tell the truth about boundaries** - encryption is opt-in (not default), contexts aren't authorization, and there's no embedding search unless paired with one.
+9. **Keep the common case tiny** - `state.value = 42` is a complete, valid program.
 
 ---