@asaidimu/utils-workspace 2.0.0 → 2.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (6) hide show
  1. package/README.md +283 -249
  2. package/index.d.mts +676 -204
  3. package/index.d.ts +676 -204
  4. package/index.js +1 -1
  5. package/index.mjs +1 -1
  6. package/package.json +5 -1
package/README.md CHANGED
@@ -1,13 +1,13 @@
1
- # @asaidimu/utils-workspace
1
+ # AI Workspace Manager
2
2
 
3
- Content-addressed workspace and conversation management for AI applications.
3
+ [![npm version](https://img.shields.io/badge/version-0.1.0-blue)](https://www.npmjs.com/package/@asaidimu/utils-workspace)
4
+ [![license](https://img.shields.io/badge/license-MIT-green)](LICENSE)
5
+ [![build status](https://img.shields.io/badge/build-passing-brightgreen)]()
6
+ [![TypeScript](https://img.shields.io/badge/TypeScript-5.0-blue)]()
4
7
 
5
- [![npm version](https://img.shields.io/npm/v/@asaidimu/utils-workspace.svg)](https://www.npmjs.com/package/@asaidimu/utils-workspace)
6
- [![License](https://img.shields.io/npm/l/@asaidimu/utils-workspace.svg)](https://github.com/asaidimu/erp-utils/blob/main/src/workspace/LICENSE)
7
- [![TypeScript](https://img.shields.io/badge/TypeScript-5.0-blue)](https://www.typescriptlang.org/)
8
- [![Vitest](https://img.shields.io/badge/tested%20with-Vitest-6e9f18)](https://vitest.dev/)
8
+ > TypeScript library for managing AI conversation workspaces — content-addressed blob storage, turn‑based transcripts, role/persona management, and prompt assembly with token budgeting.
9
9
 
10
- ## 📚 Table of Contents
10
+ ## 📖 Table of Contents
11
11
 
12
12
  - [Overview & Features](#overview--features)
13
13
  - [Installation & Setup](#installation--setup)
@@ -20,20 +20,23 @@ Content-addressed workspace and conversation management for AI applications.
20
20
 
21
21
  ## Overview & Features
22
22
 
23
- `@asaidimu/utils-workspace` is a TypeScript library that powers AI‑assisted workspaces. It provides a complete solution for managing content‑addressed binary blobs, conversation transcripts with branching and editing, workspace state (roles, preferences, context), and token‑aware prompt assembly. The library is built to be **backend‑agnostic** (supports both in‑memory and IndexedDB persistence) and **offline‑first** by default.
23
+ AI Workspace Core is a modular, offline‑first library designed to power AI‑assisted applications. It provides a complete workspace model where every conversation (session) is a DAG of **turns** (messages), backed by **content‑addressed blob storage** for large files (images, PDFs, etc.) and a flexible **context & preference** system scoped by topics.
24
24
 
25
- Why this library? Modern AI applications need to handle large binary attachments (images, documents) efficiently, support non‑linear conversations (branching, editing), and respect token budgets when sending prompts to LLMs. `@asaidimu/utils-workspace` solves these challenges with a content‑addressed storage model, a turn DAG, and a pluggable prompt builder.
25
+ The library separates **pure state updates** (reducer) from **asynchronous persistence** (ContentStore), making it suitable for reactive UIs (React, Vue, Svelte) and server environments alike. It includes built‑in IndexedDB and in‑memory backends, as well as a **PromptBuilder** that assembles prompts respecting token budgets, resolves blob references, and handles preference conflicts.
26
26
 
27
27
  ### Key Features
28
28
 
29
- - **Content‑Addressed Blob Storage** – Store blobs by SHA‑256 hash; automatic deduplication and reference counting.
30
- - **Blob Lifecycle Management** – Ref counting with lazy eviction and explicit garbage collection. Record remote file IDs after upload to providers.
31
- - **Turn Tree & Session Management** – Maintain conversation trees with parent/child relationships, edits, and branching. Auto‑link parent turns when appending.
32
- - **Dirty Buffer & Auto‑Flush** – Buffer unsaved turns in memory and flush when a size threshold is reached or on session close.
33
- - **Token‑Aware Prompt Building** – Build prompts with a pluggable token planner, context retriever (Jaccard similarity + freshness), and summarizer. Allocate budget across instructions, persona, preferences, transcript, and context.
34
- - **Pluggable Components** – Swap context retriever, token planner, summarizer, and assembler to fit your use case.
35
- - **Multi‑Backend Support** – `MemoryStorage` / `MemoryBlobStorage` for development/testing, `IndexedDBStorage` / `IndexedDBBlobStorage` for persistent browser storage.
36
- - **Workspace Commands & Reducer** – Manage index state via a pure reducer; side effects are handled by `WorkspaceManager`.
29
+ - **Blob Storage** – Content‑addressed (SHA‑256) storage with reference counting, remote ID mapping, and atomic registration. Supports `IndexedDBBlobStorage` (persistent) and `MemoryBlobStorage` (testing).
30
+ - **Session & Turn DAG** – Each session maintains a directed acyclic graph of turns (messages). Turns can be edited (new version), branched, or deleted (subtree removal). Active chain resolves to a linear transcript.
31
+ - **Role/Persona System** – Define reusable roles (e.g., “Analyst”, “Dev”) with a system persona and default preference IDs.
32
+ - **Topic‑scoped Preferences & Context** – Preferences (user instructions) and context (injected knowledge) are attached to topics. The system resolves effective preferences for a session by merging role defaults with session‑specific overrides and filtering by session topics.
33
+ - **Prompt Builder** – Three‑stage assembly:
34
+ - **Retrieval & Ranking** – Jaccard similarity + freshness weighting selects relevant context entries.
35
+ - **Planning** – Token‑aware planner respects budget, drops low‑priority items, and handles blob token estimation.
36
+ - **Assembly** – Resolves blobs (inline or remote), injects synthetic turns for truncation notices and referential attachments, and outputs a ready‑to‑use `Prompt`.
37
+ - **Reducer Pattern** – Pure `workspaceReducer` produces `DeepPartial<Workspace>` patches for all commands. WorkspaceManager applies side effects and merges async results (e.g., blob SHA‑256).
38
+ - **Multi‑backend Persistence** – Uses `@asaidimu/anansi` Database interface. WorkspaceDatabase adapts it with core schemas (Role, Preference, Context, Session, Turn). Turn DAG is stored as flat documents; the graph is reconstructed in memory.
39
+ - **Offline‑First & Eager/Lazy GC** – Blobs can be evicted eagerly (on refCount → 0) or lazily (via `gc()`). Sessions buffer turns and flush asynchronously.
37
40
 
38
41
  ---
39
42
 
@@ -41,206 +44,278 @@ Why this library? Modern AI applications need to handle large binary attachments
41
44
 
42
45
  ### Prerequisites
43
46
 
44
- - Node.js 18+ or a modern browser with IndexedDB support.
45
- - TypeScript 5.0+ (if using in a TS project).
47
+ - Node.js 18+ or modern browser (IndexedDB API required for persistent storage)
48
+ - npm, bun, or yarn
46
49
 
47
- ### Installation
50
+ ### Install
48
51
 
49
52
  ```bash
50
53
  npm install @asaidimu/utils-workspace
54
+ # or
55
+ bun add @asaidimu/utils-workspace
56
+ # or
57
+ yarn add @asaidimu/utils-workspace
51
58
  ```
52
59
 
53
60
  ### Configuration
54
61
 
55
- The library works out of the box with in‑memory storage. For persistence, set up IndexedDB backends:
62
+ The library does not require global configuration. You instantiate backends and stores directly.
56
63
 
57
64
  ```typescript
58
- import { IndexedDBStorage, IndexedDBBlobStorage, ContentStore, BlobStore, WorkspaceManager } from '@asaidimu/utils-workspace';
65
+ import {
66
+ createWorkspaceDatabase,
67
+ IndexedDBBlobStorage,
68
+ ContentStore,
69
+ WorkspaceManager
70
+ } from '@asaidimu/utils-workspace';
71
+ import { createIndexedDBDatabase } from '@asaidimu/utils-database'; // example IDB adapter
72
+
73
+ // 1. Create database (IndexedDB)
74
+ const idb = createIndexedDBDatabase({ dbName: 'my-app', version: 1 });
75
+ const db = createWorkspaceDatabase(idb);
76
+
77
+ // 2. Create blob storage (persistent)
78
+ const blobStorage = new IndexedDBBlobStorage({ dbName: 'my-app-blobs' });
79
+
80
+ // 3. ContentStore (caches, blob store, turn tree)
81
+ const contentStore = await ContentStore.create(db, blobStorage);
82
+
83
+ // 4. WorkspaceManager (dispatches commands)
84
+ const workspaceManager = new WorkspaceManager(contentStore);
85
+ ```
59
86
 
60
- // Persistent content storage
61
- const contentBackend = new IndexedDBStorage({ dbName: 'my-workspace' });
62
- await contentBackend.open();
87
+ ### Verification
63
88
 
64
- // Persistent blob storage
65
- const blobBackend = new IndexedDBBlobStorage({ dbName: 'my-workspace-blobs' });
66
- await blobBackend.open();
89
+ Run a quick command to ensure everything works:
67
90
 
68
- // Create stores with custom config
69
- const contentStore = new ContentStore(contentBackend, {
70
- cache: { roles: 10, preferences: 50, contextEntries: 20 },
71
- flush: { maxBufferSize: 20, flushIntervalMs: 30000 },
72
- });
73
-
74
- const blobStore = new BlobStore(blobBackend, { eagerEviction: false });
75
- await blobStore.init();
91
+ ```typescript
92
+ import { createWorkspace, merge } from '@asaidimu/utils-workspace';
76
93
 
77
- const manager = new WorkspaceManager(contentStore);
78
- ```
94
+ let workspace = createWorkspace({
95
+ name: "example-project",
96
+ owner: "user",
97
+ language: "en"
98
+ });
79
99
 
80
- ### Verification
100
+ const result = await workspaceManager.dispatch(workspace, {
101
+ type: 'role:add',
102
+ timestamp: new Date().toISOString(),
103
+ payload: { name: 'assistant', label: 'Assistant', persona: 'You are helpful.', preferences: [] }
104
+ });
81
105
 
82
- ```typescript
83
- console.log(manager instanceof WorkspaceManager); // true
106
+ if (result.ok) {
107
+ workspace = merge(workspace, result.value);
108
+ console.log('Role added:', workspace.index.roles['assistant']);
109
+ }
84
110
  ```
85
111
 
86
112
  ---
87
113
 
88
114
  ## Usage Documentation
89
115
 
90
- ### Basic Usage
91
-
92
- Create a workspace, add a role, create a session, and append a turn.
116
+ ### Basic Example – Create a Session, Add Turns, Build a Prompt
93
117
 
94
118
  ```typescript
95
- import { emptyIndexState, merge, WorkspaceManager, MemoryStorage, ContentStore } from '@asaidimu/utils-workspace';
96
-
97
- let workspace = {
98
- id: 'ws-1',
99
- settings: { language: 'en', defaultRole: 'assistant' },
100
- project: { name: 'My Project', owner: 'me' },
101
- index: emptyIndexState(),
102
- };
103
-
104
- const backend = new MemoryStorage();
105
- const contentStore = new ContentStore(backend);
106
- const manager = new WorkspaceManager(contentStore);
107
-
108
- // Add a role
109
- const roleCmd = {
110
- type: 'role:add',
111
- timestamp: new Date().toISOString(),
112
- payload: {
113
- id: 'r1',
114
- name: 'assistant',
115
- label: 'Assistant',
116
- persona: 'You are a helpful assistant.',
117
- preferences: [],
118
- },
119
- };
120
- const roleResult = await manager.dispatch(workspace, roleCmd);
121
- if (roleResult.ok) workspace = merge(workspace, roleResult.value);
119
+ import {
120
+ emptyWorkspace, merge,
121
+ WorkspaceManager, ContentStore,
122
+ MemoryBlobStorage, createWorkspaceDatabase,
123
+ SessionManager, PromptBuilder
124
+ } from '@asaidimu/utils-workspace';
125
+ import { createMemoryDatabase } from '@asaidimu/utils-database';
126
+
127
+ async function demo() {
128
+ // ---- Setup ----
129
+ const memDb = createMemoryDatabase();
130
+ const db = createWorkspaceDatabase(memDb);
131
+
132
+ const blobStorage = new MemoryBlobStorage();
133
+
134
+ const contentStore = await ContentStore.create(db, blobStorage);
135
+
136
+ const workspaceManager = new WorkspaceManager(contentStore);
137
+ const sessionManager = new SessionManager(workspaceManager, contentStore);
138
+
139
+ let workspace = emptyWorkspace();
140
+ const dispatch = async (cmd: any) => {
141
+ const r = await workspaceManager.dispatch(workspace, cmd);
142
+ if (r.ok) workspace = merge(workspace, r.value);
143
+ else throw new Error(r.error.code);
144
+ };
145
+
146
+ // ---- Seed data ----
147
+ await dispatch({
148
+ type: 'role:add', timestamp: new Date().toISOString(),
149
+ payload: { name: 'analyst', label: 'Analyst', persona: 'You are a financial analyst.', preferences: [] }
150
+ });
151
+ await dispatch({
152
+ type: 'preference:add', timestamp: new Date().toISOString(),
153
+ payload: { id: 'pref1', content: 'Use KES for currency', topics: ['finance'], timestamp: new Date().toISOString() }
154
+ });
155
+ await dispatch({
156
+ type: 'context:add', timestamp: new Date().toISOString(),
157
+ payload: { key: 'kb:q4', content: { kind: 'text', value: 'Q4 revenue: KES 4.2M' }, topics: ['finance'], timestamp: new Date().toISOString() }
158
+ });
159
+ await dispatch({
160
+ type: 'session:create', timestamp: new Date().toISOString(),
161
+ payload: { id: 'session-1', label: 'Earnings Call', role: 'analyst', topics: ['finance'], preferences: ['pref1'] }
162
+ });
163
+
164
+ // ---- Open session, add a turn ----
165
+ const openResult = await sessionManager.open(workspace, 'session-1');
166
+ if (!openResult.ok) throw new Error('open failed');
167
+ const { session } = openResult.value;
168
+
169
+ const addResult = await session.addTurn(workspace, {
170
+ id: 'turn-1', version: 0, role: 'user',
171
+ blocks: [{ type: 'text', text: 'What was the Q4 revenue?' }],
172
+ timestamp: new Date().toISOString(), parent: null
173
+ });
174
+ if (addResult.ok) workspace = merge(workspace, addResult.value);
175
+ await session.flush();
176
+
177
+ // ---- Resolve effective session & build prompt ----
178
+ const resolved = await workspaceManager.resolveSession(workspace, 'session-1');
179
+ if (!resolved.ok) throw new Error('resolve failed');
180
+
181
+ const promptBuilder = new PromptBuilder({ blobResolver: contentStore.getBlobResolver() });
182
+ const prompt = await promptBuilder.build(resolved.value, {
183
+ tokenBudget: { total: 4000 },
184
+ relevanceConfig: { recentMessageWindow: 5 }
185
+ });
186
+
187
+ console.log(prompt.system.persona); // "You are a financial analyst."
188
+ console.log(prompt.system.preferences[0].content); // "Use KES for currency"
189
+ console.log(prompt.transcript.turns[0].blocks[0].text); // "What was the Q4 revenue?"
190
+ console.log(prompt.budget.used); // token count
191
+ }
122
192
 
123
- // Create a session
124
- const sessionCmd = {
125
- type: 'session:create',
126
- timestamp: new Date().toISOString(),
127
- payload: { id: 's1', label: 'First Chat', role: 'assistant', topics: [] },
128
- };
129
- const sessionResult = await manager.dispatch(workspace, sessionCmd);
130
- if (sessionResult.ok) workspace = merge(workspace, sessionResult.value);
193
+ demo();
131
194
  ```
132
195
 
133
- ### Session Management (with SessionManager)
196
+ ### CLI Commands
134
197
 
135
- `SessionManager` provides a live `Session` object with turn tree operations.
198
+ AI Workspace Core is a **library**, not a CLI. However, you can build your own CLI around its commands. The core command types are:
136
199
 
137
- ```typescript
138
- import { SessionManager } from '@asaidimu/utils-workspace';
200
+ | Command Family | Example |
201
+ |----------------|---------|
202
+ | Role | `role:add`, `role:update`, `role:delete` |
203
+ | Preference | `preference:add`, `preference:update`, `preference:delete` |
204
+ | Context | `context:add`, `context:update`, `context:delete` |
205
+ | Session | `session:create`, `session:fork`, `session:delete`, `session:role:switch`, `session:topics:add`, `session:preferences:override` |
206
+ | Turn | `turn:add`, `turn:edit`, `turn:branch`, `turn:delete` |
207
+ | Blob | `blob:register`, `blob:retain`, `blob:release`, `blob:purge`, `blob:record_remote_id` |
139
208
 
140
- const sessionManager = new SessionManager(manager, contentStore);
141
- const openResult = await sessionManager.open(workspace, 's1');
142
- if (!openResult.ok) throw new Error('Failed to open session');
209
+ ### API Reference
143
210
 
144
- const { session } = openResult.value;
211
+ #### Core Classes
145
212
 
146
- // Add a turn (auto‑links parent to current head)
147
- const turn = {
148
- id: crypto.randomUUID(),
149
- version: 0,
150
- role: 'user',
151
- blocks: [{ type: 'text', text: 'Hello, world!' }],
152
- timestamp: new Date().toISOString(),
153
- parent: null,
154
- };
155
- const addResult = await session.addTurn(workspace, turn);
156
- if (addResult.ok) workspace = merge(workspace, addResult.value);
213
+ - **`BlobStore`** – Manages blob registration, ref counting, remote IDs, and resolution. Created internally by `ContentStore`.
214
+ - **`ContentStore`** – High‑level API for all entities (Role, Preference, Context, Session, Turn) and blob operations. Caches records.
215
+ - **`WorkspaceManager`** – Dispatches commands, reduces state, and persists side effects. Entry point for state changes.
216
+ - **`SessionManager`** – Opens/closes live sessions, loads turn DAG, returns `Session` objects.
217
+ - **`Session`** – Live session with methods: `addTurn`, `editTurn`, `branchFrom`, `deleteTurn`, `switchVersionLeft/Right`, `resolve`, `flush`, `dispose`.
218
+ - **`TurnTree`** – Low‑level DAG persistence (append, branch, delete subtree). Used by `Session` via `ContentStore`.
219
+ - **`PromptBuilder`** – Assembles prompts from `EffectiveSession` with token budgeting and blob resolution.
157
220
 
158
- // Flush buffered turns to storage
159
- await session.flush();
221
+ #### Key Types
160
222
 
161
- // Close session when done
162
- await sessionManager.close(session);
163
- ```
223
+ - `Workspace` – Contains `id`, `settings`, `project`, and `index` (in‑memory read model).
224
+ - `BlobRef` – Lightweight pointer to a blob (sha256, mediaType, sizeBytes, optional filename).
225
+ - `ResolvedBlob` – Either `{ kind: 'inline', data: Uint8Array }` or `{ kind: 'remote', fileId, providerId }`.
226
+ - `EffectiveSession` – Resolved session with role, preferences, context, transcript, and optional instructions.
227
+ - `Prompt` – Ready to send to an LLM; includes system block, context, transcript turns, budget breakdown, warnings, and conflicts.
164
228
 
165
- ### Blob Registration & Resolution
229
+ ### Configuration Examples
166
230
 
167
- ```typescript
168
- import { BlobStore, MemoryBlobStorage } from '@asaidimu/utils-workspace';
231
+ #### Customising Token Estimator
169
232
 
170
- const blobStorage = new MemoryBlobStorage();
171
- const blobStore = new BlobStore(blobStorage);
172
- await blobStore.init();
173
-
174
- const data = new TextEncoder().encode('Content of a document');
175
- const registerResult = await blobStore.register(data, 'text/plain', 'readme.txt');
176
- if (registerResult.ok) {
177
- const blobRef = registerResult.value;
178
- // Use blobRef in a turn or context block
179
- }
233
+ ```typescript
234
+ const promptBuilder = new PromptBuilder({
235
+ blobResolver: contentStore.getBlobResolver(),
236
+ planner: new DefaultTokenPlanner() // or extend
237
+ });
180
238
 
181
- // Later, resolve the blob for a prompt
182
- const resolveResult = await blobStore.resolveRef(blobRef, null);
183
- if (resolveResult.ok) {
184
- const resolved = resolveResult.value;
185
- if (resolved.kind === 'inline') {
186
- console.log(new TextDecoder().decode(resolved.data));
239
+ await promptBuilder.build(session, {
240
+ tokenBudget: {
241
+ total: 8000,
242
+ estimator: (text: string) => Math.ceil(text.length / 3), // custom
243
+ blobTokensPerKB: 0.5
187
244
  }
188
- }
245
+ });
189
246
  ```
190
247
 
191
- ### Building a Prompt
248
+ #### Using a Different Context Retriever
192
249
 
193
- `PromptBuilder` assembles a token‑aware prompt from an `EffectiveSession`.
250
+ Implement `ContextRetriever` interface:
194
251
 
195
252
  ```typescript
196
- import { PromptBuilder } from '@asaidimu/utils-workspace';
197
-
198
- const builder = new PromptBuilder({
199
- blobResolver: blobStore.resolveRefs.bind(blobStore),
200
- });
201
-
202
- const effective = await manager.resolveSession(workspace, 's1');
203
- if (!effective.ok) throw new Error('Failed to resolve session');
253
+ class MyRetriever implements ContextRetriever {
254
+ rank(input: ContextRankingInput): Context[] {
255
+ // custom logic
256
+ }
257
+ }
204
258
 
205
- const prompt = await builder.build(effective.value, {
206
- tokenBudget: { total: 4000 },
207
- relevanceConfig: { recentMessageWindow: 5, minScore: 0.2 },
208
- providerId: 'anthropic', // for remote blobs
259
+ const promptBuilder = new PromptBuilder({
260
+ blobResolver: contentStore.getBlobResolver(),
261
+ retriever: new MyRetriever()
209
262
  });
210
-
211
- console.log(prompt.system.instructions);
212
- console.log(prompt.transcript.turns);
213
- console.log(prompt.warnings);
214
263
  ```
215
264
 
216
265
  ### Common Use Cases
217
266
 
218
- #### 1. Offline‑First Chat Application
267
+ #### 1. Upload an Image and Attach to a Turn
219
268
 
220
- - Use `IndexedDBStorage` and `IndexedDBBlobStorage` for persistence.
221
- - Set `eagerEviction: false` to keep blobs until manually cleaned.
222
- - Let `ContentStore` auto‑flush when `maxBufferSize` is reached.
223
- - Call `blobStore.gc()` occasionally to reclaim space.
269
+ ```typescript
270
+ // Register blob via command (async)
271
+ const registerCmd = {
272
+ type: 'blob:register',
273
+ timestamp: new Date().toISOString(),
274
+ payload: { data: imageUint8Array, mediaType: 'image/png', filename: 'chart.png' }
275
+ };
276
+ const regResult = await workspaceManager.dispatch(workspace, registerCmd);
277
+ if (regResult.ok) workspace = merge(workspace, regResult.value);
278
+ const blobRef = regResult.value.index.blobs[sha256]; // BlobRef
224
279
 
225
- #### 2. Multi‑Session Workspace with Topic‑Based Context
280
+ // Add turn referencing the blob
281
+ const turn = {
282
+ id: crypto.randomUUID(), version: 0, role: 'user',
283
+ blocks: [{ type: 'image', ref: blobRef, altText: 'Revenue chart' }],
284
+ timestamp: new Date().toISOString(), parent: null
285
+ };
286
+ await session.addTurn(workspace, turn);
287
+ ```
226
288
 
227
- - Create context entries with topics.
228
- - Use `session:topics:add` command to associate topics with a session.
229
- - `resolveSession` automatically pulls in context entries that match those topics.
289
+ #### 2. Edit a Previous Turn (Versioning)
230
290
 
231
- #### 3. Branching Conversations
291
+ ```typescript
292
+ const editResult = await session.editTurn(workspace, 'turn-1', [
293
+ { type: 'text', text: 'Corrected: Q4 revenue was KES 4.5M' }
294
+ ]);
295
+ // A new version (1) is created. Session head moves to version 1 of turn-1.
296
+ ```
232
297
 
233
- - Use `session.branchFrom()` to fork a conversation tree.
234
- - Each branch maintains its own head pointer.
235
- - Fork an entire session with the `session:fork` command.
298
+ #### 3. Branch a Conversation
236
299
 
237
- #### 4. Integrating with AI Providers
300
+ ```typescript
301
+ const branchTurn = {
302
+ id: crypto.randomUUID(), version: 0, role: 'user',
303
+ blocks: [{ type: 'text', text: 'Let’s explore a different scenario.' }],
304
+ timestamp: new Date().toISOString(),
305
+ parent: { id: 'turn-1', version: 0 } // branch from original version
306
+ };
307
+ await session.branchFrom(workspace, branchTurn);
308
+ // New head points to branchTurn, original chain remains.
309
+ ```
238
310
 
239
- - After uploading a blob to a provider (e.g., Anthropic), record the remote ID:
240
- ```typescript
241
- await blobStore.recordRemoteId(sha256, 'anthropic', 'file_abc123');
242
- ```
243
- - In `PromptBuilder.build`, pass the `providerId` to get remote references instead of inline bytes.
311
+ #### 4. Garbage Collect Unreferenced Blobs
312
+
313
+ ```typescript
314
+ // lazy eviction (default) – only delete bytes, keep record
315
+ const deletedCount = await contentStore.blobs.gc();
316
+ // full purge – delete bytes AND record
317
+ const purgedCount = await contentStore.blobs.gcFull();
318
+ ```
244
319
 
245
320
  ---
246
321
 
@@ -248,83 +323,45 @@ console.log(prompt.warnings);
248
323
 
249
324
  ### Core Components
250
325
 
251
- - **`WorkspaceManager`** – Facade that coordinates the pure reducer, content side effects, and session resolution.
252
- - **`ContentStore`** – Manages roles, preferences, context, and transcript windows. Handles LRU caching, dirty buffers, and session activation.
253
- - **`BlobStore`** – Owns the blob registry (ref counts, remote IDs) and SHA‑256 computation. Interacts with a `BlobStorage` backend.
254
- - **`TurnTree`** – Pure logic for turn chains, subtree deletion, and head management. Used by `ContentStore` and `Session`.
255
- - **`Session`** – Live object for an open session. Holds the turn DAG and dirty buffer, provides mutation methods (`addTurn`, `editTurn`, `branchFrom`, `deleteTurn`, `switchLeft`/`Right`).
256
- - **`SessionManager`** – Entry point for opening and closing sessions; loads the turn DAG from storage.
257
- - **`PromptBuilder`** – Orchestrates context retrieval, token planning, blob resolution, and final assembly.
258
- - **Storage backends** – `MemoryStorage`, `IndexedDBStorage` (content), `MemoryBlobStorage`, `IndexedDBBlobStorage` (blobs).
326
+ - **BlobStorage** – Interface for raw bytes + record persistence. Implementations: `IndexedDBBlobStorage`, `MemoryBlobStorage`.
327
+ - **BlobStore** – Adds ref counting, remote ID mapping, and resolution. Maintains in‑memory cache of `BlobRecord`s.
328
+ - **ContentStore** – Owns `TurnTree` and `BlobStore`. Provides high‑level methods for all entities. Caches roles/preferences/context with LRU.
329
+ - **TurnTree** – Flat storage of `Turn` documents (sessionId, id, version). Reconstructs DAG in memory (`buildNodeGraph`). Handles `append`, `replaceVersion`, `branch`, `deleteSubtree`, and head pointer on `SessionMeta`.
330
+ - **Reducer** – Pure function `workspaceReducer(workspace, command) => DeepPartial<Workspace>`. Updates `Index` summaries (roles, preferences, context, sessions, topics, blobs).
331
+ - **WorkspaceManager** – Orchestrates reducer + side effects. Dispatches commands, merges patches, exposes `resolveSession`.
332
+ - **SessionManager** – Opens sessions by loading turn DAG via `TurnTree.buildNodeGraph()`. Returns `Session` object.
333
+ - **Session** – Live DAG with mutable `nodes` and dirty buffer. Provides turn mutations that update memory, buffer, and trigger flushes.
334
+ - **PromptBuilder** – Retrieves relevant context, plans token usage, resolves blob refs, assembles final `Prompt`.
259
335
 
260
336
  ### Data Flow
261
337
 
262
- 1. **User action** → `WorkspaceManager.dispatch(command)`.
263
- 2. **Reducer** produces a `DeepPartial<Workspace>` patch and schedules side effects.
264
- 3. **Side effects** write to `ContentStore` (roles, preferences, context, turns via `TurnTree`).
265
- 4. **Session activation** loads the turn DAG via `TurnTree.buildNodeGraph()`.
266
- 5. **Turn mutations** update the in‑memory `Session.nodes` and dirty buffer; auto‑flush writes to storage when buffer size exceeds threshold.
267
- 6. **Prompt building** retrieves an `EffectiveSession` (via `Session.resolve` or `ContentStore.resolveSession`), ranks context, plans token usage, resolves blobs, and assembles the final `Prompt`.
268
-
269
- ### Extension Points
270
-
271
- - **`ContextRetriever`** – Replace the default Jaccard‑based retriever with a vector or hybrid approach.
272
- - **`TokenPlanner`** – Implement custom token estimation (e.g., using tiktoken) and priority strategies.
273
- - **`Summarizer`** – Hook into an LLM to compress transcripts.
274
- - **`BlobStorage` / `ContentStorage`** – Add new backends (e.g., S3, SQLite) by implementing the interface.
275
-
276
- ---
277
-
278
- ## Development & Contributing
279
-
280
- ### Development Setup
281
-
282
- 1. Clone the repository:
283
- ```bash
284
- git clone https://github.com/asaidimu/erp-utils.git
285
- cd erp-utils/src/workspace
286
- ```
287
- 2. Install dependencies:
288
- ```bash
289
- npm install
290
- ```
291
- 3. Build the project:
292
- ```bash
293
- npm run build
294
- ```
295
-
296
- ### Scripts
297
-
298
- | Script | Description |
299
- |----------------------|-------------------------------------------|
300
- | `npm test` | Run tests once (Vitest) |
301
- | `npm run test:watch` | Run tests in watch mode |
302
- | `npm run test:browser` | Run tests in a browser environment |
303
-
304
- ### Testing
305
-
306
- The test suite uses `vitest` and `fake-indexeddb/auto` to simulate IndexedDB. Run the tests with:
307
-
308
- ```bash
309
- npm test
338
+ ```
339
+ Command → WorkspaceManager.dispatch()
340
+ ├─ workspaceReducer() → pure patch
341
+ ├─ handleContentSideEffects() → persist via ContentStore, return extra patch
342
+ └─ merge patches → return to caller
343
+
344
+ Session mutations (addTurn, editTurn, etc.):
345
+ Session (in‑memory nodes + dirtyBuffer)
346
+ └─ on flush: TurnTree.appendBatch(sessionId, turns, finalHead)
347
+ └─ updates SessionMeta.head
348
+ └─ stores Turn documents
349
+
350
+ Prompt building:
351
+ EffectiveSession (resolved by ContentStore)
352
+ └─ PromptBuilder.build()
353
+ ├─ ContextRetriever.rank()
354
+ ├─ TokenPlanner.plan()
355
+ ├─ BlobStore.resolveRefs()
356
+ └─ PromptAssembler.assemble()
310
357
  ```
311
358
 
312
- All core functionality is covered. When contributing, please add tests for new features or bug fixes.
313
-
314
- ### Contributing Guidelines
315
-
316
- - **Branching**: Use feature branches off `main`.
317
- - **Commit messages**: Follow conventional commits (e.g., `feat: add new retriever`).
318
- - **Pull requests**: Ensure tests pass and add new tests for changes.
319
- - **Code style**: The project uses Prettier and ESLint. Run formatting before committing.
320
-
321
- ### Issue Reporting
359
+ ### Extension Points
322
360
 
323
- Please open an issue on [GitHub](https://github.com/asaidimu/erp-utils/issues) with:
324
- - A clear description of the problem.
325
- - Steps to reproduce.
326
- - Expected and actual behaviour.
327
- - Version of the library and environment (Node.js / browser).
361
+ - **Custom BlobStorage** – Implement `BlobStorage` interface to use S3, OPFS, etc.
362
+ - **Custom ContextRetriever** – Provide alternative relevance algorithms.
363
+ - **Custom TokenPlanner** – Override budget allocation logic.
364
+ - **Summarizer** – Inject summarizer to compress long transcripts before prompt assembly.
328
365
 
329
366
  ---
330
367
 
@@ -332,30 +369,27 @@ Please open an issue on [GitHub](https://github.com/asaidimu/erp-utils/issues) w
332
369
 
333
370
  ### Troubleshooting
334
371
 
335
- | Issue | Solution |
336
- |--------------------------------------------|--------------------------------------------------------------------------------------------|
337
- | `Database not open` error | Call `open()` on the storage backend before using it. |
338
- | Blob not found in prompt | Ensure the blob is still referenced (refCount > 0) and not evicted. Run `gc()` only when safe. |
339
- | Turns missing after reload | Check that `flush()` was called before deactivation, or that `maxBufferSize` was reached. |
340
- | Prompt builder returns warnings for blobs | Verify that the blob is registered and, if remote, that `recordRemoteId` was called. |
372
+ | Problem | Likely cause | Solution |
373
+ |---------|--------------|----------|
374
+ | `Blob bytes not found locally` | Blob evicted but still referenced. | Re‑register the blob or disable eager eviction. |
375
+ | `Database not open. Call open() first.` | Forgot to call `await blobStorage.open()`. | Ensure `open()` is called before any operation. |
376
+ | `Constraint failed: unique index by_session_id_ver` | Duplicate (sessionId, id, version). | Check that `Turn` IDs are unique per session. |
377
+ | `PromptBuilder` fails with `BLOB_ERROR` | Blob resolution failed (remote ID missing or bytes missing). | Verify remote mapping or ensure blob is still present. |
341
378
 
342
379
  ### FAQ
343
380
 
344
- **Q: How does blob deduplication work?**
345
- A: `BlobStore` computes SHA‑256 of the data. If a blob with the same hash already exists, only the ref count is incremented – bytes are never duplicated.
381
+ **Q: Can I use this library in a browser without IndexedDB?**
382
+ A: Yes – use `MemoryBlobStorage` and an in‑memory database adapter. However, data will not persist across page reloads.
346
383
 
347
- **Q: Can I use this library in a Node.js environment without IndexedDB?**
348
- A: Yes. Use `MemoryStorage` and `MemoryBlobStorage` for in‑memory persistence. For long‑term storage, you would need to implement a custom backend (e.g., file system or SQLite).
384
+ **Q: How do I export a workspace for backup?**
385
+ A: Use `IndexedDBBlobStorage.exportAllBytes()` and `ContentStore` methods to dump all records. There is also a `WorkspaceBundle` type for structured export.
349
386
 
350
- **Q: How do I manage token budgets for large transcripts?**
351
- A: Use the `summarizer` option in `PromptBuilder` to compress older turns. The default `TokenPlanner` trims from the oldest turns when the budget is exceeded.
387
+ **Q: Does the library support streaming large files?**
388
+ A: Not directly. Blobs are stored as complete `Uint8Array`. For very large files, consider splitting or using a remote storage provider and storing only the remote ID.
352
389
 
353
- **Q: What happens if I delete a role that is used by a session?**
354
- A: The reducer will reject the command with `INVALID_COMMAND`. You must first switch sessions to a different role or delete the sessions.
390
+ **Q: Can I use a different LLM provider’s file API?**
391
+ A: Yes – after uploading a blob to a provider (e.g., Anthropic), call `recordBlobRemoteId(sha256, providerId, fileId)`. The `resolveRef` method will return a remote blob reference for that provider.
355
392
 
356
393
  ### License
357
- This project is licensed under the [MIT License](https://github.com/asaidimu/erp-utils/blob/main/LICENSE.md).
358
-
359
- ### Acknowledgments
360
394
 
361
- Built with inspiration from content‑addressed storage systems (IPFS, git) and conversation tree models. Special thanks to the open‑source community for tools like TypeScript, Vitest, and fake-indexeddb.
395
+ [MIT](https://github.com/asaidimu/erp-utils/blob/main/LICENSE) © Saidimu