memorio 5.1.2 → 5.1.4

package/AGENTS.md

@@ -69,9 +69,7 @@ If the code you're generating is client-side (a single browser tab, one user), `
   outside a guaranteed browser context must guard: `if (idb.db.support())` or check
   `memorio.getCapabilities().hasIndexedDB` - don't rely on the no-op warning alone reaching a log
   anyone will see.
-- **`sqlite` persistence serializes the entire database on every flush** - not incremental. Never
-  generate code that calls a persisting write inside a loop; batch writes in one transaction/run
-  and persist once after the batch, per the README's guidance.
+- **`sqlite` persistence uses journal + checkpoint (incremental writes, periodic full snapshots).** Batch writes and persist once after the batch per the README's guidance - calling `persist()` inside a tight loop still forces expensive checkpoints.
 - **`observer('state.some.path', cb)` paths are plain strings, unchecked against `state`'s actual
   shape.** A typo or a later rename fails *silently* - the observer just never fires again. Prefer
   `memorio.typed<T>()` + `registerSchema()` for anything you'd hate to have silently stop working,

package/CHANGELOG.md

@@ -0,0 +1,83 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+## [Unreleased]
+
+### Added
+- **SQLite journal + checkpoint persistence**: When `persistence: true` is set on `sqlite.db.create()` (or via `sqlite.config({ persistence: true })`), writes are recorded in an append-only journal with HLC timestamps. Periodic checkpoints (every 50 ops or 250 ms of write-idle) snapshot the full database to `store`. On reopen, the checkpoint is loaded and pending journal entries are replayed - no writes lost between checkpoints. New methods: `sqlite.db.flush()`, `sqlite.db.persist()`, `sqlite.db.journal` namespace (`append`, `entries`, `count`, `clear`, `replay`, `checkpoint`), `sqlite.db.journal` for direct journal inspection.
+
+### Fixed
+- **`package.json` `sideEffects`**: Added `./core/global.ts` to the `sideEffects` array so that esbuild/tsup no longer tree-shakes the `import './core/global'` side-effect-only import in `index.ts`. Without this fix, the `core/global.ts` initialization block (which defines `globalThis.memorio`) was eliminated from production builds, causing `memorio.global()` to throw `Cannot read properties of undefined (reading 'global')`.
+
+## [5.1.2] - 2026-09-14
+
+### Fixed
+- **`types/encryption.d.ts`**: Fixed parse error (`TS1005: ',' expected`) in `deriveKey` signature - missing comma after `alg` parameter caused all consumers' type checkers to fail, even with `skipLibCheck: true`.
+- **`core/platform.ts`**: `_isBun()` now checks `typeof globalAny.Bun.version === 'string'` instead of `typeof globalAny.Bun.sqlite === 'function'`, correctly detecting the Bun runtime so `hasBunSqlite()` returns `true` and the `bun:sqlite` backend loads.
+- **`functions/sqlite/tools/bun-sqlite-adapter.ts`**: Removed unused `_index` variable.
+- **`functions/security/index.ts`**: Removed redundant type re-export block that conflicted with inline `export type`/`export interface` declarations under `isolatedModules`.
+- **`extension/vscode/src/astIndexer.ts`**: Removed unused imports (`path`, `StateUsage`, `StatePathInfo`, `StoreOperation`) and unused `lines`/`program` parameters.
+- **`extension/vscode/esbuild.config.mjs`**: Replaced TypeScript type annotations with JSDoc `/** @type */` to fix parse errors in `.mjs` context.
+
+### Packaging
+- **`package.json`**: Changed `"sideEffects": true` to `"sideEffects": ["./global.js", "./global.cjs"]` so tree-shaking works for consumers using only `state`/`store` - previously the entire engine (SQLite, memory, encryption) was bundled unconditionally.
+- Added `types/bun-sqlite.d.ts` module declaration so `import('bun:sqlite')` resolves under TypeScript.
+
+### Migration guide: globals change (v5 breaking change)
+
+**Upgrading from 4.x:** the globals (`state`, `store`, `session`, `cache`, `idb`, `observer`, `useObserver`) are no longer installed by `import "memorio"`. Applications that use them without importing them must change one line:
+
+```ts
+// 4.x - globals installed on import
+import 'memorio'
+
+// 5.x - globals installed via the global entrypoint
+import 'memorio/global'
+```
+
+Nothing else changes. The named imports (`import { state } from 'memorio'`) work identically.
+
+**Bundler cache:** after upgrading, restart the dev server and clear `node_modules/.vite` (Vite) or `.next/cache` (Next.js), because a stale pre-bundle can mask the breakage - the app may appear to work and then fail at the next dependency re-optimisation.
+
+## [5.1.0] - 2026-09-12
+
+### Added
+- **Bun runtime support**: `memorio.isBun()` and `memorio.getCapabilities().hasBunSqlite` for runtime detection alongside `isBrowser()`/`isNode()`/`isDeno()`/`isEdge()`.
+- **`bun:sqlite` backend**: SQLite operations under Bun use the native `bun:sqlite` binding - no WASM, real incremental writes, no full-database serialization on flush. Falls back to `sql.js` (browser) where Bun sqlite is unavailable. No changes to consumers: the same `sqlite.db.create()`, `sqlite.query.run()`, `sqlite.query.select()` APIs work unchanged.
+- **Bun dev toolchain**: `bunfig.toml`, `test:bun`, `build:bun`, `lint:bun`, `tsc:bun` scripts. Existing `npm`/`npx` scripts remain as fallbacks. `bun` added to `devDependencies` only (not `dependencies`/`peerDependencies`).
+- **`security.sanitizeValue()`**: recursive sanitization with HTML entity decode, URL decode, XSS scheme blocking, and `__proto__`/`constructor`/`prototype` key stripping.
+- **`security.RBAC`**: `addAccessRule()`, `hasPermission()` with regex/function patterns, fail-closed by default when no rule matches.
+- **`security.audit logging`**: `setAuditLogger()`, `logAudit()`, `isAuditActive()`. `hasPermission()` auto-logs denied attempts when an audit logger is configured.
+- **`computed` module**: lazy-memoized derived state with dependency tracking (`compute()`, `getValue()`, `subscribe()`, `remove()`, `list()`, `clear()`).
+- **`broadcast` module**: cross-tab sync via `BroadcastChannel` + `storage` event fallback (`install()`, `uninstall()`, `isInstalled()`).
+- **PBKDF2 600k iterations** (NIST SP 800-132), up from 100k. `deriveKey()` accepts `iterations` parameter.
+- **`encryption.exportKey()` / `encryption.importKey()`**: base64 key export/import with length validation.
+- **`store.config()` / `session.config()`**: accept `onError`, `maxObjectSize`, `maxTotalSize` alongside existing `encryptionKey`.
+
+### Security
+- PBKDF2 key derivation upgraded to 600,000 iterations (NIST SP 800-132 recommendation).
+- `sanitizeValue` now strips dangerous URL schemes (`javascript:`, `vbscript:`, `data:text/html`, `about:blank`, `chrome:`) without word-boundary requirements, catching encoded variants like `%6javascript:`.
+- `sanitizeValue` now handles `<object>`, `<embed>`, `<svg>`, `<iframe>`, `<script>` tag removal including void elements and nested payloads.
+- `sanitizeValue` skips `__proto__`, `constructor`, `prototype` keys to prevent prototype pollution.
+- `encryption.importKey()` validates AES-256 key length (32 bytes) and throws descriptive error for truncated input.
+- RBAC `hasPermission()` fail-closed: no matching rule → denied + auto-audit-logged.
+- `AuditEntry.action` type extended to include `'read'`, `'write'`, `'admin'`, `'permission_check'`.
+
+### Documentation
+- README restyled: tagline "The memory layer for AI agents and apps - owned by the user, not the vendor."
+- README: "Application memory" section moved before "Global or explicit".
+- README: "Skip this unless..." callout lines added to 10 skippable sections.
+- README: layers table gained "When" column (Start here / Add later).
+- README: Cross-platform table gained Bun column with `bun:sqlite` native support.
+- README: Quick reading map added after Install.
+- README: "This page covers everything... Most apps only need the first three sections." callout after first example.
+- SECURITY.md: documented Web Crypto async/non-blocking behavior.
+- SECURITY.md: PBKDF2 iteration count updated to 600k.
+- SECURITY-HARDENING.md: checklist expanded to 16 items.
+- CHANGELOG.md: this file.
+
+### Developer experience
+- Lint check test (`lint-check.test.ts`) integrated into test suite.
+- `tests/bun/` directory for Bun-specific tests (`bun:test` imports).
+- `tests/vitest/tests/` for Node.js/Vitest tests (unchanged).

package/README.md

@@ -4,7 +4,6 @@
 
 [![npm version](https://img.shields.io/npm/v/memorio)](https://www.npmjs.com/package/memorio)
 [![license](https://img.shields.io/npm/l/memorio)](./LICENSE)
-[![bundle size](https://img.shields.io/bundlephobia/minzip/memorio)](https://bundlephobia.com/package/memorio)
 [![Socket Badge](https://socket.dev/api/badge/npm/package/memorio)](https://socket.dev/npm/package/memorio)
 [![Known Vulnerabilities](https://snyk.io/test/npm/memorio/badge.svg)](https://snyk.io/test/npm/memorio)
 [![zero deps](https://img.shields.io/badge/dependencies-0-brightgreen)](#security)
@@ -13,14 +12,10 @@
 ![Edge Workers](https://img.shields.io/badge/Edge%20Workers-compatible-gray)
 ![TypeScript](https://img.shields.io/badge/TypeScript-native-gray?logo=typescript)
 ![React](https://img.shields.io/badge/React-compatible-gray?logo=react)
-![Tests](https://img.shields.io/badge/tests-500+%20passed-green)
 [![license](https://img.shields.io/npm/l/memorio.svg)](#license)
 
-<!--
-[![CI](https://img.shields.io/github/actions/workflow/status/GITHUB_ORG/GITHUB_REPO/ci.yml?branch=main)](https://github.com/GITHUB_ORG/GITHUB_REPO/actions)
--->
 
-**The memory layer for AI agents and apps — owned by the user, not the vendor.**
+**The memory layer for AI agents and apps - owned by the user, not the vendor.**
 
 ```ts
 import 'memorio/global'
@@ -41,13 +36,13 @@ Just data, available where your application needs it.
 
 That's the whole API for the simple case. No provider tree, no boilerplate.
 
-But real apps grow. Most AI-powered apps (and most apps in general) eventually need more than "just state" — persistence, session data, caches, structured browser storage, local SQL, application memory, history, optional sync. Usually that means pulling in a different library — and a different mental model — for each. Memorio gives them one consistent runtime, without making the simple case complicated:
+But real apps grow. Most AI-powered apps (and most apps in general) eventually need more than "just state" - persistence, session data, caches, structured browser storage, local SQL, application memory, history, optional sync. Usually that means pulling in a different library - and a different mental model - for each. Memorio gives them one consistent runtime, without making the simple case complicated:
 
 ```ts
 state.value = 42
 ```
 
-But the reason memorio exists isn't just "fewer dependencies." It's this: **the memory an AI agent builds about a user shouldn't be locked inside a vendor's server, opaque, and impossible to inspect or move.** Memorio's memory layer is structured, inspectable, editable, and — when you turn on encryption — readable only with a key the vendor doesn't hold. It runs client-side, in the browser, on the edge, or in Node.
+But the reason memorio exists isn't just "fewer dependencies." It's this: **the memory an AI agent builds about a user shouldn't be locked inside a vendor's server, opaque, and impossible to inspect or move.** Memorio's memory layer is structured, inspectable, editable, and - when you turn on encryption - readable only with a key the vendor doesn't hold. It runs client-side, in the browser, on the edge, or in Node.
 
 That said, this covers *structured* memory - preferences, decisions, facts with confidence and provenance - not semantic similarity search. If your agent needs "find things like this conversation," pair memorio with a dedicated vector store; see [`markdown/MEMORY.md`](./markdown/MEMORY.md).
 
@@ -63,6 +58,20 @@ npm install memorio
 
 Zero production dependencies in core. Optional integrations (`react`, `sql.js`, etc.) are installed only when you use them.
 
+### Globals migration (v5)
+
+If you are upgrading from memorio 4.x and your code uses `state`, `store`, `session`, `cache`, `observer`, or `useObserver` as **bare globals** (without importing them), change one line:
+
+```ts
+// 4.x
+import 'memorio'
+
+// 5.x
+import 'memorio/global'
+```
+
+Named imports (`import { state } from 'memorio'`) are unaffected. After upgrading, restart your dev server and clear `node_modules/.vite` / `.next/cache` to avoid stale pre-bundles masking the change. See [`markdown/IMPORT.md`](./markdown/IMPORT.md) for full details.
+
 ---
 
 ## Start here
@@ -70,17 +79,27 @@ Zero production dependencies in core. Optional integrations (`react`, `sql.js`,
 Three things cover most apps - reactive state, persistence, and observation:
 
 ```ts
-import { state, persist, observer } from 'memorio'
+import { state, observer } from 'memorio'
 
 state.user = { name: 'Sara' }
 
-persist('state.user')
+const stopPersisting = state.persist('state.user')
 
 observer('state.user', user => {
   console.log(user)
 })
 ```
 
+`state.persist()` creates a bidirectional mirror between `state` (live in-memory)
+and `store` (durable copy). It seeds the initial value from `store` if present,
+then every write to that path flows to `store` automatically. The return value is
+an unsubscribe function - call it to stop the mirroring (the value already in
+`store` is preserved):
+
+```ts
+stopPersisting() // from here, state.user is in-memory only; store no longer updated
+```
+
 And this is what makes memorio a *memory* layer, not just a state manager - structured, inspectable facts about a user or agent, not just ephemeral UI state:
 
 ```ts
@@ -148,7 +167,6 @@ For most apps, that's enough. Everything below is an optional capability you can
 ### Other
 | Topic | Reference |
 | --- | --- |
-| Version history | [`CHANGELOG.md`](./CHANGELOG.md) |
 | Working examples - each file has a `Run:` comment at the top (typically `npx ts-node examples/<name>.ts`, or `npx ts-node --esm examples/<name>.tsx` for React examples) | [`examples/`](./examples/) |
 | Architecture decisions | [`adr/`](./adr/) |
 
@@ -159,9 +177,9 @@ For most apps, that's enough. Everything below is an optional capability you can
 Memorio is upfront about what it doesn't do, so you don't find out the hard way:
 
 - **No encryption by default.** Data is stored in plain text unless you explicitly turn on `memorio.encryption` or pass an `encryptionKey`. See [`SECURITY.md`](./SECURITY.md).
-- **Namespaces/contexts aren't a security boundary.** `memorio.isolate()` gives you logical separation between tenants or sessions, not authorization or access control — enforce that at your application layer.
-- **SQLite persistence (`sql.js` backend) isn't incremental.** Each write exports and re-saves the whole database, which gets costly as data grows. The `bun:sqlite` backend avoids this cost but doesn't support `export()` at all — see [`markdown/SQLITE.md`](./markdown/SQLITE.md).
-- **`memory.context()` is rule-based, not embedding-based.** It ranks structured entries by recency, confidence, and type — it does not do semantic similarity search. Pair it with a vector store if your agent needs "find things like this."
+- **Namespaces/contexts aren't a security boundary.** `memorio.isolate()` gives you logical separation between tenants or sessions, not authorization or access control - enforce that at your application layer.
+- **SQLite persistence uses a journal + checkpoint strategy.** Individual writes are recorded in an append-only journal (incremental); periodic checkpoints snapshot the full database. This avoids serializing the entire database on every write. The `bun:sqlite` backend doesn't support `export()` but handles persistence natively - see [`markdown/SQLITE.md`](./markdown/SQLITE.md).
+- **`memory.context()` is rule-based, not embedding-based.** It ranks structured entries by recency, confidence, and type - it does not do semantic similarity search. Pair it with a vector store if your agent needs "find things like this."
 - **Key management is your responsibility.** Memorio's encryption is client-side; it doesn't manage, rotate, or store keys for you.
 
 ---

package/SUMMARY.md

@@ -17,7 +17,7 @@
 | `react-observer.tsx` | `useObserver` in due modalità (auto-discovery vs deps espliciti) combinato con `typed<T>()` e `registerSchema()`. |
 | `semantic-memory.ts` | Memoria applicativa per un'app LLM-backed: remember/update con confidence e source, retrieval con `memory.context()` - con nota esplicita che non è ricerca semantica per embedding. |
 | `session-advanced.ts` | Uso avanzato di `session`: auth token, bozza di form, carrello, dimensione dello storage. |
-| `sqlite-batched-writes.ts` | Come evitare di serializzare l'intero DB SQLite ad ogni riga: batch di insert seguito da un solo flush. |
+| `sqlite-batched-writes.ts` | Con il journal+checkpoint, i write sono incrementali ma i checkpoint (snapshot completo) sono costosi: batch di insert seguito da un solo flush. |
 | `state-advanced.ts` | Stato annidato, array, locking di un valore (`.lock()`), path tracking, rimozione stato. |
 | `store-advanced.ts` | `store` avanzato: persistenza, quota, alias dei metodi, gestione errori, serializzazione di vari tipi. |
 | `sync.ts` *(nuovo)* | Local-first sync: configurazione di un `SyncProvider` (push/pull/resolve), scope `device`/`user`/`shared`, ispezione e replay del journal. |
@@ -47,7 +47,7 @@
 | `SCHEMA.md` | Validazione runtime dei percorsi di stato: schema oggetto/array/enum/funzione custom. |
 | `SECURITY.md` | Postura di sicurezza del progetto (minacce coperte, cosa NON fa Memorio, come viene gestito l'accesso ai dati). |
 | `SESSION.md` | Reference di `session` (sessionStorage) con fallback in-memory fuori dal browser. |
-| `SQLITE.md` | Reference del layer SQLite via `sql.js`: caricamento lazy, persistenza, query. |
+| `SQLITE.md` | Reference del layer SQLite via `sql.js`/`bun:sqlite`: caricamento lazy, persistenza journal+checkpoint, query. |
 | `STATE.md` | Reference dello stato reattivo basato su Proxy - il layer centrale di Memorio. |
 | `STORE.md` | Reference di `store` (localStorage) con fallback in-memory fuori dal browser. |
 | `SYNC.md` | Sincronizzazione local-first opzionale: scope, journal, conflict resolution, strategie avanzate per multi-device (HLC, tombstones, fractional indexing). |

package/examples/sqlite-batched-writes.ts

@@ -1,57 +1,60 @@
-/**
- * sqlite-batched-writes.ts
- *
- * Scenario: importing or writing many rows into memorio's sqlite layer.
- *
- * Persistence there serializes the ENTIRE database on every flush - not
- * incremental. Calling a persisting write inside a per-row loop is the
- * single most common way to accidentally make this layer slow.
- */
-import { memorio } from 'memorio'
-
-interface UserRow {
-  id: number
-  name: string
-  role: string
-}
-
-export async function importUsers(users: UserRow[]) {
-  await memorio.sqlite.ready
-  await memorio.sqlite.db.create('app', { persistence: true })
-
-  await memorio.sqlite.query.run(
-    'app',
-    `CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT NOT NULL, role TEXT)`
-  )
-
-  // Bad: this would serialize the whole DB after every single insert.
-  //
-  // for (const user of users) {
-  //   await memorio.sqlite.query.run('app', 'INSERT INTO users ...', [...])
-  //   await memorio.sqlite.db.flush('app') // <- don't do this per row
-  // }
-
-  // Good: run every insert first, persist once after the batch.
-  for (const user of users) {
-    await memorio.sqlite.query.run(
-      'app',
-      `INSERT INTO users (id, name, role) VALUES (?, ?, ?)`,
-      [user.id, user.name, user.role]
-    )
-  }
-
-  // Persist explicitly, once, after the whole batch - check the installed
-  // version's API for the exact flush/persist call name if it differs from
-  // automatic persistence-on-close.
-  const admins = await memorio.sqlite.query.select(
-    'app',
-    `SELECT * FROM users WHERE role = ?`,
-    ['admin']
-  )
-
-  return admins
-}
-
-// For read-heavy or scratch-space use where durability doesn't matter,
-// skip { persistence: true } entirely - sqlite runs in memory by default,
-// which avoids the serialize cost altogether.
+/**
+ * sqlite-batched-writes.ts
+ *
+ * Scenario: importing or writing many rows into memorio's sqlite layer.
+ *
+ * With persistence enabled, each write is recorded in an append-only journal
+ * (incremental). Periodic checkpoints - every 50 operations or 250 ms of
+ * write-idle - take a full database snapshot. Calling `flush()` inside a
+ * per-row loop triggers a checkpoint on every iteration, which is expensive.
+ */
+import { memorio } from 'memorio'
+
+interface UserRow {
+  id: number
+  name: string
+  role: string
+}
+
+export async function importUsers(users: UserRow[]) {
+  await memorio.sqlite.ready
+  await memorio.sqlite.db.create('app', { persistence: true })
+
+  await memorio.sqlite.query.run(
+    'app',
+    `CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT NOT NULL, role TEXT)`
+  )
+
+  // Bad: this would trigger a full checkpoint after every single insert.
+  //
+  // for (const user of users) {
+  //   await memorio.sqlite.query.run('app', 'INSERT INTO users ...', [...])
+  //   await memorio.sqlite.db.flush('app') // <- don't do this per row
+  // }
+
+  // Good: run every insert first, flush once after the batch. Writes are
+  // journaled incrementally (cheap); the checkpoint (full snapshot) happens
+  // once at the end.
+  for (const user of users) {
+    await memorio.sqlite.query.run(
+      'app',
+      `INSERT INTO users (id, name, role) VALUES (?, ?, ?)`,
+      [user.id, user.name, user.role]
+    )
+  }
+
+  // Flush explicitly, once, after the whole batch - check the installed
+  // version's API for the exact flush/persist call name if it differs from
+  // automatic persistence-on-close.
+  const admins = await memorio.sqlite.query.select(
+    'app',
+    `SELECT * FROM users WHERE role = ?`,
+    ['admin']
+  )
+
+  return admins
+}
+
+// For read-heavy or scratch-space use where durability doesn't matter,
+// skip { persistence: true } entirely - sqlite runs in memory by default,
+// which avoids journal and checkpoint overhead altogether.