memorio 5.1.1 → 5.1.3

package/AGENTS.md

@@ -69,9 +69,7 @@ If the code you're generating is client-side (a single browser tab, one user), `
   outside a guaranteed browser context must guard: `if (idb.db.support())` or check
   `memorio.getCapabilities().hasIndexedDB` - don't rely on the no-op warning alone reaching a log
   anyone will see.
-- **`sqlite` persistence serializes the entire database on every flush** - not incremental. Never
-  generate code that calls a persisting write inside a loop; batch writes in one transaction/run
-  and persist once after the batch, per the README's guidance.
+- **`sqlite` persistence uses journal + checkpoint (incremental writes, periodic full snapshots).** Batch writes and persist once after the batch per the README's guidance — calling `persist()` inside a tight loop still forces expensive checkpoints.
 - **`observer('state.some.path', cb)` paths are plain strings, unchecked against `state`'s actual
   shape.** A typo or a later rename fails *silently* - the observer just never fires again. Prefer
   `memorio.typed<T>()` + `registerSchema()` for anything you'd hate to have silently stop working,

package/CHANGELOG.md

@@ -0,0 +1,80 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+## [Unreleased]
+
+### Added
+- **SQLite journal + checkpoint persistence**: When `persistence: true` is set on `sqlite.db.create()` (or via `sqlite.config({ persistence: true })`), writes are recorded in an append-only journal with HLC timestamps. Periodic checkpoints (every 50 ops or 250 ms of write-idle) snapshot the full database to `store`. On reopen, the checkpoint is loaded and pending journal entries are replayed — no writes lost between checkpoints. New methods: `sqlite.db.flush()`, `sqlite.db.persist()`, `sqlite.db.journal` namespace (`append`, `entries`, `count`, `clear`, `replay`, `checkpoint`), `sqlite.db.journal` for direct journal inspection.
+
+## [5.1.2] - 2026-09-14
+
+### Fixed
+- **`types/encryption.d.ts`**: Fixed parse error (`TS1005: ',' expected`) in `deriveKey` signature — missing comma after `alg` parameter caused all consumers' type checkers to fail, even with `skipLibCheck: true`.
+- **`core/platform.ts`**: `_isBun()` now checks `typeof globalAny.Bun.version === 'string'` instead of `typeof globalAny.Bun.sqlite === 'function'`, correctly detecting the Bun runtime so `hasBunSqlite()` returns `true` and the `bun:sqlite` backend loads.
+- **`functions/sqlite/tools/bun-sqlite-adapter.ts`**: Removed unused `_index` variable.
+- **`functions/security/index.ts`**: Removed redundant type re-export block that conflicted with inline `export type`/`export interface` declarations under `isolatedModules`.
+- **`extension/vscode/src/astIndexer.ts`**: Removed unused imports (`path`, `StateUsage`, `StatePathInfo`, `StoreOperation`) and unused `lines`/`program` parameters.
+- **`extension/vscode/esbuild.config.mjs`**: Replaced TypeScript type annotations with JSDoc `/** @type */` to fix parse errors in `.mjs` context.
+
+### Packaging
+- **`package.json`**: Changed `"sideEffects": true` to `"sideEffects": ["./global.js", "./global.cjs"]` so tree-shaking works for consumers using only `state`/`store` — previously the entire engine (SQLite, memory, encryption) was bundled unconditionally.
+- Added `types/bun-sqlite.d.ts` module declaration so `import('bun:sqlite')` resolves under TypeScript.
+
+### Migration guide: globals change (v5 breaking change)
+
+**Upgrading from 4.x:** the globals (`state`, `store`, `session`, `cache`, `idb`, `observer`, `useObserver`) are no longer installed by `import "memorio"`. Applications that use them without importing them must change one line:
+
+```ts
+// 4.x — globals installed on import
+import 'memorio'
+
+// 5.x — globals installed via the global entrypoint
+import 'memorio/global'
+```
+
+Nothing else changes. The named imports (`import { state } from 'memorio'`) work identically.
+
+**Bundler cache:** after upgrading, restart the dev server and clear `node_modules/.vite` (Vite) or `.next/cache` (Next.js), because a stale pre-bundle can mask the breakage — the app may appear to work and then fail at the next dependency re-optimisation.
+
+## [5.1.0] - 2026-09-12
+
+### Added
+- **Bun runtime support**: `memorio.isBun()` and `memorio.getCapabilities().hasBunSqlite` for runtime detection alongside `isBrowser()`/`isNode()`/`isDeno()`/`isEdge()`.
+- **`bun:sqlite` backend**: SQLite operations under Bun use the native `bun:sqlite` binding - no WASM, real incremental writes, no full-database serialization on flush. Falls back to `sql.js` (browser) where Bun sqlite is unavailable. No changes to consumers: the same `sqlite.db.create()`, `sqlite.query.run()`, `sqlite.query.select()` APIs work unchanged.
+- **Bun dev toolchain**: `bunfig.toml`, `test:bun`, `build:bun`, `lint:bun`, `tsc:bun` scripts. Existing `npm`/`npx` scripts remain as fallbacks. `bun` added to `devDependencies` only (not `dependencies`/`peerDependencies`).
+- **`security.sanitizeValue()`**: recursive sanitization with HTML entity decode, URL decode, XSS scheme blocking, and `__proto__`/`constructor`/`prototype` key stripping.
+- **`security.RBAC`**: `addAccessRule()`, `hasPermission()` with regex/function patterns, fail-closed by default when no rule matches.
+- **`security.audit logging`**: `setAuditLogger()`, `logAudit()`, `isAuditActive()`. `hasPermission()` auto-logs denied attempts when an audit logger is configured.
+- **`computed` module**: lazy-memoized derived state with dependency tracking (`compute()`, `getValue()`, `subscribe()`, `remove()`, `list()`, `clear()`).
+- **`broadcast` module**: cross-tab sync via `BroadcastChannel` + `storage` event fallback (`install()`, `uninstall()`, `isInstalled()`).
+- **PBKDF2 600k iterations** (NIST SP 800-132), up from 100k. `deriveKey()` accepts `iterations` parameter.
+- **`encryption.exportKey()` / `encryption.importKey()`**: base64 key export/import with length validation.
+- **`store.config()` / `session.config()`**: accept `onError`, `maxObjectSize`, `maxTotalSize` alongside existing `encryptionKey`.
+
+### Security
+- PBKDF2 key derivation upgraded to 600,000 iterations (NIST SP 800-132 recommendation).
+- `sanitizeValue` now strips dangerous URL schemes (`javascript:`, `vbscript:`, `data:text/html`, `about:blank`, `chrome:`) without word-boundary requirements, catching encoded variants like `%6javascript:`.
+- `sanitizeValue` now handles `<object>`, `<embed>`, `<svg>`, `<iframe>`, `<script>` tag removal including void elements and nested payloads.
+- `sanitizeValue` skips `__proto__`, `constructor`, `prototype` keys to prevent prototype pollution.
+- `encryption.importKey()` validates AES-256 key length (32 bytes) and throws descriptive error for truncated input.
+- RBAC `hasPermission()` fail-closed: no matching rule → denied + auto-audit-logged.
+- `AuditEntry.action` type extended to include `'read'`, `'write'`, `'admin'`, `'permission_check'`.
+
+### Documentation
+- README restyled: tagline "The memory layer for AI agents and apps - owned by the user, not the vendor."
+- README: "Application memory" section moved before "Global or explicit".
+- README: "Skip this unless..." callout lines added to 10 skippable sections.
+- README: layers table gained "When" column (Start here / Add later).
+- README: Cross-platform table gained Bun column with `bun:sqlite` native support.
+- README: Quick reading map added after Install.
+- README: "This page covers everything... Most apps only need the first three sections." callout after first example.
+- SECURITY.md: documented Web Crypto async/non-blocking behavior.
+- SECURITY.md: PBKDF2 iteration count updated to 600k.
+- SECURITY-HARDENING.md: checklist expanded to 16 items.
+- CHANGELOG.md: this file.
+
+### Developer experience
+- Lint check test (`lint-check.test.ts`) integrated into test suite.
+- `tests/bun/` directory for Bun-specific tests (`bun:test` imports).
+- `tests/vitest/tests/` for Node.js/Vitest tests (unchanged).

package/README.md

@@ -63,6 +63,20 @@ npm install memorio
 
 Zero production dependencies in core. Optional integrations (`react`, `sql.js`, etc.) are installed only when you use them.
 
+### Globals migration (v5)
+
+If you are upgrading from memorio 4.x and your code uses `state`, `store`, `session`, `cache`, `observer`, or `useObserver` as **bare globals** (without importing them), change one line:
+
+```ts
+// 4.x
+import 'memorio'
+
+// 5.x
+import 'memorio/global'
+```
+
+Named imports (`import { state } from 'memorio'`) are unaffected. After upgrading, restart your dev server and clear `node_modules/.vite` / `.next/cache` to avoid stale pre-bundles masking the change. See [`markdown/IMPORT.md`](./markdown/IMPORT.md) for full details.
+
 ---
 
 ## Start here
@@ -148,7 +162,6 @@ For most apps, that's enough. Everything below is an optional capability you can
 ### Other
 | Topic | Reference |
 | --- | --- |
-| Version history | [`CHANGELOG.md`](./CHANGELOG.md) |
 | Working examples - each file has a `Run:` comment at the top (typically `npx ts-node examples/<name>.ts`, or `npx ts-node --esm examples/<name>.tsx` for React examples) | [`examples/`](./examples/) |
 | Architecture decisions | [`adr/`](./adr/) |
 
@@ -160,7 +173,7 @@ Memorio is upfront about what it doesn't do, so you don't find out the hard way:
 
 - **No encryption by default.** Data is stored in plain text unless you explicitly turn on `memorio.encryption` or pass an `encryptionKey`. See [`SECURITY.md`](./SECURITY.md).
 - **Namespaces/contexts aren't a security boundary.** `memorio.isolate()` gives you logical separation between tenants or sessions, not authorization or access control — enforce that at your application layer.
-- **SQLite persistence (`sql.js` backend) isn't incremental.** Each write exports and re-saves the whole database, which gets costly as data grows. The `bun:sqlite` backend avoids this cost but doesn't support `export()` at all — see [`markdown/SQLITE.md`](./markdown/SQLITE.md).
+- **SQLite persistence uses a journal + checkpoint strategy.** Individual writes are recorded in an append-only journal (incremental); periodic checkpoints snapshot the full database. This avoids serializing the entire database on every write. The `bun:sqlite` backend doesn't support `export()` but handles persistence natively — see [`markdown/SQLITE.md`](./markdown/SQLITE.md).
 - **`memory.context()` is rule-based, not embedding-based.** It ranks structured entries by recency, confidence, and type — it does not do semantic similarity search. Pair it with a vector store if your agent needs "find things like this."
 - **Key management is your responsibility.** Memorio's encryption is client-side; it doesn't manage, rotate, or store keys for you.
 

package/SUMMARY.md

@@ -17,7 +17,7 @@
 | `react-observer.tsx` | `useObserver` in due modalità (auto-discovery vs deps espliciti) combinato con `typed<T>()` e `registerSchema()`. |
 | `semantic-memory.ts` | Memoria applicativa per un'app LLM-backed: remember/update con confidence e source, retrieval con `memory.context()` - con nota esplicita che non è ricerca semantica per embedding. |
 | `session-advanced.ts` | Uso avanzato di `session`: auth token, bozza di form, carrello, dimensione dello storage. |
-| `sqlite-batched-writes.ts` | Come evitare di serializzare l'intero DB SQLite ad ogni riga: batch di insert seguito da un solo flush. |
+| `sqlite-batched-writes.ts` | Con il journal+checkpoint, i write sono incrementali ma i checkpoint (snapshot completo) sono costosi: batch di insert seguito da un solo flush. |
 | `state-advanced.ts` | Stato annidato, array, locking di un valore (`.lock()`), path tracking, rimozione stato. |
 | `store-advanced.ts` | `store` avanzato: persistenza, quota, alias dei metodi, gestione errori, serializzazione di vari tipi. |
 | `sync.ts` *(nuovo)* | Local-first sync: configurazione di un `SyncProvider` (push/pull/resolve), scope `device`/`user`/`shared`, ispezione e replay del journal. |
@@ -47,7 +47,7 @@
 | `SCHEMA.md` | Validazione runtime dei percorsi di stato: schema oggetto/array/enum/funzione custom. |
 | `SECURITY.md` | Postura di sicurezza del progetto (minacce coperte, cosa NON fa Memorio, come viene gestito l'accesso ai dati). |
 | `SESSION.md` | Reference di `session` (sessionStorage) con fallback in-memory fuori dal browser. |
-| `SQLITE.md` | Reference del layer SQLite via `sql.js`: caricamento lazy, persistenza, query. |
+| `SQLITE.md` | Reference del layer SQLite via `sql.js`/`bun:sqlite`: caricamento lazy, persistenza journal+checkpoint, query. |
 | `STATE.md` | Reference dello stato reattivo basato su Proxy - il layer centrale di Memorio. |
 | `STORE.md` | Reference di `store` (localStorage) con fallback in-memory fuori dal browser. |
 | `SYNC.md` | Sincronizzazione local-first opzionale: scope, journal, conflict resolution, strategie avanzate per multi-device (HLC, tombstones, fractional indexing). |

package/examples/sqlite-batched-writes.ts

@@ -3,9 +3,10 @@
  *
  * Scenario: importing or writing many rows into memorio's sqlite layer.
  *
- * Persistence there serializes the ENTIRE database on every flush - not
- * incremental. Calling a persisting write inside a per-row loop is the
- * single most common way to accidentally make this layer slow.
+ * With persistence enabled, each write is recorded in an append-only journal
+ * (incremental). Periodic checkpoints — every 50 operations or 250 ms of
+ * write-idle — take a full database snapshot. Calling `flush()` inside a
+ * per-row loop triggers a checkpoint on every iteration, which is expensive.
  */
 import { memorio } from 'memorio'
 
@@ -24,14 +25,16 @@ export async function importUsers(users: UserRow[]) {
     `CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT NOT NULL, role TEXT)`
   )
 
-  // Bad: this would serialize the whole DB after every single insert.
+  // Bad: this would trigger a full checkpoint after every single insert.
   //
   // for (const user of users) {
   //   await memorio.sqlite.query.run('app', 'INSERT INTO users ...', [...])
   //   await memorio.sqlite.db.flush('app') // <- don't do this per row
   // }
 
-  // Good: run every insert first, persist once after the batch.
+  // Good: run every insert first, flush once after the batch. Writes are
+  // journaled incrementally (cheap); the checkpoint (full snapshot) happens
+  // once at the end.
   for (const user of users) {
     await memorio.sqlite.query.run(
       'app',
@@ -40,7 +43,7 @@ export async function importUsers(users: UserRow[]) {
     )
   }
 
-  // Persist explicitly, once, after the whole batch - check the installed
+  // Flush explicitly, once, after the whole batch — check the installed
   // version's API for the exact flush/persist call name if it differs from
   // automatic persistence-on-close.
   const admins = await memorio.sqlite.query.select(
@@ -54,4 +57,4 @@ export async function importUsers(users: UserRow[]) {
 
 // For read-heavy or scratch-space use where durability doesn't matter,
 // skip { persistence: true } entirely - sqlite runs in memory by default,
-// which avoids the serialize cost altogether.
+// which avoids journal and checkpoint overhead altogether.