@asaidimu/utils-workspace 1.0.1 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +214 -296
- package/index.d.mts +308 -276
- package/index.d.ts +308 -276
- package/index.js +1 -1
- package/index.mjs +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
# @asaidimu/utils-workspace
|
|
2
2
|
|
|
3
|
-
Content-addressed workspace and conversation management for AI applications.
|
|
3
|
+
Content-addressed workspace and conversation management for AI applications.
|
|
4
4
|
|
|
5
5
|
[](https://www.npmjs.com/package/@asaidimu/utils-workspace)
|
|
6
|
-
[](https://github.com/asaidimu/erp-utils/blob/main/src/workspace/LICENSE)
|
|
7
|
+
[](https://www.typescriptlang.org/)
|
|
8
|
+
[](https://vitest.dev/)
|
|
9
9
|
|
|
10
|
-
## Table of Contents
|
|
10
|
+
## 📚 Table of Contents
|
|
11
11
|
|
|
12
12
|
- [Overview & Features](#overview--features)
|
|
13
13
|
- [Installation & Setup](#installation--setup)
|
|
@@ -20,20 +20,20 @@ Content-addressed workspace and conversation management for AI applications. Han
|
|
|
20
20
|
|
|
21
21
|
## Overview & Features
|
|
22
22
|
|
|
23
|
-
|
|
23
|
+
`@asaidimu/utils-workspace` is a TypeScript library that powers AI‑assisted workspaces. It provides a complete solution for managing content‑addressed binary blobs, conversation transcripts with branching and editing, workspace state (roles, preferences, context), and token‑aware prompt assembly. The library is built to be **backend‑agnostic** (supports both in‑memory and IndexedDB persistence) and **offline‑first** by default.
|
|
24
24
|
|
|
25
|
-
|
|
25
|
+
Why this library? Modern AI applications need to handle large binary attachments (images, documents) efficiently, support non‑linear conversations (branching, editing), and respect token budgets when sending prompts to LLMs. `@asaidimu/utils-workspace` solves these challenges with a content‑addressed storage model, a turn DAG, and a pluggable prompt builder.
|
|
26
26
|
|
|
27
27
|
### Key Features
|
|
28
28
|
|
|
29
|
-
- **Content
|
|
30
|
-
- **
|
|
31
|
-
- **
|
|
32
|
-
- **
|
|
33
|
-
- **Prompt Building**
|
|
34
|
-
- **
|
|
35
|
-
- **
|
|
36
|
-
- **
|
|
29
|
+
- **Content‑Addressed Blob Storage** – Store blobs by SHA‑256 hash; automatic deduplication and reference counting.
|
|
30
|
+
- **Blob Lifecycle Management** – Ref counting with lazy eviction and explicit garbage collection. Record remote file IDs after upload to providers.
|
|
31
|
+
- **Turn Tree & Session Management** – Maintain conversation trees with parent/child relationships, edits, and branching. Auto‑link parent turns when appending.
|
|
32
|
+
- **Dirty Buffer & Auto‑Flush** – Buffer unsaved turns in memory and flush when a size threshold is reached or on session close.
|
|
33
|
+
- **Token‑Aware Prompt Building** – Build prompts with a pluggable token planner, context retriever (Jaccard similarity + freshness), and summarizer. Allocate budget across instructions, persona, preferences, transcript, and context.
|
|
34
|
+
- **Pluggable Components** – Swap context retriever, token planner, summarizer, and assembler to fit your use case.
|
|
35
|
+
- **Multi‑Backend Support** – `MemoryStorage` / `MemoryBlobStorage` for development/testing, `IndexedDBStorage` / `IndexedDBBlobStorage` for persistent browser storage.
|
|
36
|
+
- **Workspace Commands & Reducer** – Manage index state via a pure reducer; side effects are handled by `WorkspaceManager`.
|
|
37
37
|
|
|
38
38
|
---
|
|
39
39
|
|
|
@@ -41,8 +41,8 @@ The system implements a command-reducer pattern where state mutations are expres
|
|
|
41
41
|
|
|
42
42
|
### Prerequisites
|
|
43
43
|
|
|
44
|
-
- Node.js 18+
|
|
45
|
-
- TypeScript 5.0+ (
|
|
44
|
+
- Node.js 18+ or a modern browser with IndexedDB support.
|
|
45
|
+
- TypeScript 5.0+ (if using in a TS project).
|
|
46
46
|
|
|
47
47
|
### Installation
|
|
48
48
|
|
|
@@ -50,55 +50,37 @@ The system implements a command-reducer pattern where state mutations are expres
|
|
|
50
50
|
npm install @asaidimu/utils-workspace
|
|
51
51
|
```
|
|
52
52
|
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
```bash
|
|
56
|
-
yarn add @asaidimu/utils-workspace
|
|
57
|
-
```
|
|
53
|
+
### Configuration
|
|
58
54
|
|
|
59
|
-
|
|
55
|
+
The library works out of the box with in‑memory storage. For persistence, set up IndexedDB backends:
|
|
60
56
|
|
|
61
57
|
```typescript
|
|
62
|
-
import {
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
WorkspaceManager,
|
|
66
|
-
PromptBuilder,
|
|
67
|
-
MemoryBlobStorage,
|
|
68
|
-
BlobStore
|
|
69
|
-
merge,
|
|
70
|
-
} from '@asaidimu/utils-workspace';
|
|
71
|
-
|
|
72
|
-
// Initialize content storage (IndexedDB for persistence)
|
|
58
|
+
import { IndexedDBStorage, IndexedDBBlobStorage, ContentStore, BlobStore, WorkspaceManager } from '@asaidimu/utils-workspace';
|
|
59
|
+
|
|
60
|
+
// Persistent content storage
|
|
73
61
|
const contentBackend = new IndexedDBStorage({ dbName: 'my-workspace' });
|
|
74
62
|
await contentBackend.open();
|
|
75
63
|
|
|
76
|
-
//
|
|
77
|
-
const blobBackend = new
|
|
78
|
-
|
|
79
|
-
await blobStore.init();
|
|
64
|
+
// Persistent blob storage
|
|
65
|
+
const blobBackend = new IndexedDBBlobStorage({ dbName: 'my-workspace-blobs' });
|
|
66
|
+
await blobBackend.open();
|
|
80
67
|
|
|
81
|
-
// Create
|
|
82
|
-
const contentStore = new ContentStore(contentBackend
|
|
68
|
+
// Create stores with custom config
|
|
69
|
+
const contentStore = new ContentStore(contentBackend, {
|
|
70
|
+
cache: { roles: 10, preferences: 50, contextEntries: 20 },
|
|
71
|
+
flush: { maxBufferSize: 20, flushIntervalMs: 30000 },
|
|
72
|
+
});
|
|
73
|
+
|
|
74
|
+
const blobStore = new BlobStore(blobBackend, { eagerEviction: false });
|
|
75
|
+
await blobStore.init();
|
|
83
76
|
|
|
84
|
-
// Create workspace manager
|
|
85
77
|
const manager = new WorkspaceManager(contentStore);
|
|
78
|
+
```
|
|
86
79
|
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
id: 'ws-1',
|
|
92
|
-
settings: { language: 'en', defaultRole: 'assistant' },
|
|
93
|
-
project: { name: 'My Project', owner: 'user' },
|
|
94
|
-
indexes: { sessions: {}, roles: {}, preferences: {}, context: {}, topics: {} }
|
|
95
|
-
},
|
|
96
|
-
roles: {},
|
|
97
|
-
blobs: {},
|
|
98
|
-
preferences: {},
|
|
99
|
-
context: {},
|
|
100
|
-
sessions: {}
|
|
101
|
-
};
|
|
80
|
+
### Verification
|
|
81
|
+
|
|
82
|
+
```typescript
|
|
83
|
+
console.log(manager instanceof WorkspaceManager); // true
|
|
102
84
|
```
|
|
103
85
|
|
|
104
86
|
---
|
|
@@ -107,239 +89,189 @@ const emptyState = {
|
|
|
107
89
|
|
|
108
90
|
### Basic Usage
|
|
109
91
|
|
|
110
|
-
Create a role, session, and
|
|
92
|
+
Create a workspace, add a role, create a session, and append a turn.
|
|
111
93
|
|
|
112
94
|
```typescript
|
|
113
|
-
|
|
95
|
+
import { emptyIndexState, merge, WorkspaceManager, MemoryStorage, ContentStore } from '@asaidimu/utils-workspace';
|
|
96
|
+
|
|
97
|
+
let workspace = {
|
|
98
|
+
id: 'ws-1',
|
|
99
|
+
settings: { language: 'en', defaultRole: 'assistant' },
|
|
100
|
+
project: { name: 'My Project', owner: 'me' },
|
|
101
|
+
index: emptyIndexState(),
|
|
102
|
+
};
|
|
103
|
+
|
|
104
|
+
const backend = new MemoryStorage();
|
|
105
|
+
const contentStore = new ContentStore(backend);
|
|
106
|
+
const manager = new WorkspaceManager(contentStore);
|
|
114
107
|
|
|
115
|
-
//
|
|
116
|
-
const
|
|
117
|
-
type: 'role:add'
|
|
108
|
+
// Add a role
|
|
109
|
+
const roleCmd = {
|
|
110
|
+
type: 'role:add',
|
|
118
111
|
timestamp: new Date().toISOString(),
|
|
119
112
|
payload: {
|
|
120
|
-
id: '
|
|
113
|
+
id: 'r1',
|
|
121
114
|
name: 'assistant',
|
|
122
115
|
label: 'Assistant',
|
|
123
116
|
persona: 'You are a helpful assistant.',
|
|
124
|
-
preferences: []
|
|
125
|
-
}
|
|
117
|
+
preferences: [],
|
|
118
|
+
},
|
|
126
119
|
};
|
|
120
|
+
const roleResult = await manager.dispatch(workspace, roleCmd);
|
|
121
|
+
if (roleResult.ok) workspace = merge(workspace, roleResult.value);
|
|
127
122
|
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
indexState = merge(indexState, roleResult.value) ;
|
|
132
|
-
}
|
|
133
|
-
|
|
134
|
-
// 2. Create a session
|
|
135
|
-
const createSessionCmd = {
|
|
136
|
-
type: 'session:create' as const,
|
|
123
|
+
// Create a session
|
|
124
|
+
const sessionCmd = {
|
|
125
|
+
type: 'session:create',
|
|
137
126
|
timestamp: new Date().toISOString(),
|
|
138
|
-
payload: {
|
|
139
|
-
id: 'session-1',
|
|
140
|
-
label: 'My First Conversation',
|
|
141
|
-
role: 'assistant',
|
|
142
|
-
topics: ['general'],
|
|
143
|
-
preferences: []
|
|
144
|
-
}
|
|
127
|
+
payload: { id: 's1', label: 'First Chat', role: 'assistant', topics: [] },
|
|
145
128
|
};
|
|
129
|
+
const sessionResult = await manager.dispatch(workspace, sessionCmd);
|
|
130
|
+
if (sessionResult.ok) workspace = merge(workspace, sessionResult.value);
|
|
131
|
+
```
|
|
146
132
|
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
133
|
+
### Session Management (with SessionManager)
|
|
134
|
+
|
|
135
|
+
`SessionManager` provides a live `Session` object with turn tree operations.
|
|
136
|
+
|
|
137
|
+
```typescript
|
|
138
|
+
import { SessionManager } from '@asaidimu/utils-workspace';
|
|
151
139
|
|
|
152
|
-
|
|
153
|
-
await
|
|
140
|
+
const sessionManager = new SessionManager(manager, contentStore);
|
|
141
|
+
const openResult = await sessionManager.open(workspace, 's1');
|
|
142
|
+
if (!openResult.ok) throw new Error('Failed to open session');
|
|
154
143
|
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
144
|
+
const { session } = openResult.value;
|
|
145
|
+
|
|
146
|
+
// Add a turn (auto‑links parent to current head)
|
|
147
|
+
const turn = {
|
|
148
|
+
id: crypto.randomUUID(),
|
|
149
|
+
version: 0,
|
|
150
|
+
role: 'user',
|
|
151
|
+
blocks: [{ type: 'text', text: 'Hello, world!' }],
|
|
158
152
|
timestamp: new Date().toISOString(),
|
|
159
|
-
|
|
160
|
-
sessionId: 'session-1',
|
|
161
|
-
turn: {
|
|
162
|
-
id: 'turn-1',
|
|
163
|
-
version: 0,
|
|
164
|
-
role: 'user',
|
|
165
|
-
blocks: [{ type: 'text', text: 'Hello, how are you?' }],
|
|
166
|
-
timestamp: new Date().toISOString(),
|
|
167
|
-
roleSnapshot: 'assistant',
|
|
168
|
-
parent: null
|
|
169
|
-
}
|
|
170
|
-
}
|
|
153
|
+
parent: null,
|
|
171
154
|
};
|
|
155
|
+
const addResult = await session.addTurn(workspace, turn);
|
|
156
|
+
if (addResult.ok) workspace = merge(workspace, addResult.value);
|
|
172
157
|
|
|
173
|
-
|
|
174
|
-
await
|
|
158
|
+
// Flush buffered turns to storage
|
|
159
|
+
await session.flush();
|
|
160
|
+
|
|
161
|
+
// Close session when done
|
|
162
|
+
await sessionManager.close(session);
|
|
175
163
|
```
|
|
176
164
|
|
|
177
|
-
###
|
|
165
|
+
### Blob Registration & Resolution
|
|
178
166
|
|
|
179
167
|
```typescript
|
|
180
|
-
|
|
181
|
-
const imageData = new TextEncoder().encode('fake-image-data');
|
|
182
|
-
const blobResult = await blobStore.register(imageData, 'image/jpeg', 'photo.jpg');
|
|
183
|
-
|
|
184
|
-
if (blobResult.ok) {
|
|
185
|
-
const blobRef = blobResult.value;
|
|
186
|
-
|
|
187
|
-
// Use the blob in a turn
|
|
188
|
-
const turnWithBlob = {
|
|
189
|
-
id: 'turn-2',
|
|
190
|
-
version: 0,
|
|
191
|
-
role: 'user',
|
|
192
|
-
blocks: [
|
|
193
|
-
{ type: 'text', text: 'What is in this image?' },
|
|
194
|
-
{ type: 'image', blob: blobRef }
|
|
195
|
-
],
|
|
196
|
-
timestamp: new Date().toISOString(),
|
|
197
|
-
roleSnapshot: 'assistant',
|
|
198
|
-
parent: null
|
|
199
|
-
};
|
|
200
|
-
|
|
201
|
-
await manager.dispatch(indexState, {
|
|
202
|
-
type: 'turn:add',
|
|
203
|
-
timestamp: new Date().toISOString(),
|
|
204
|
-
payload: { sessionId: 'session-1', turn: turnWithBlob }
|
|
205
|
-
});
|
|
206
|
-
}
|
|
207
|
-
```
|
|
168
|
+
import { BlobStore, MemoryBlobStorage } from '@asaidimu/utils-workspace';
|
|
208
169
|
|
|
209
|
-
|
|
170
|
+
const blobStorage = new MemoryBlobStorage();
|
|
171
|
+
const blobStore = new BlobStore(blobStorage);
|
|
172
|
+
await blobStore.init();
|
|
210
173
|
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
const
|
|
222
|
-
|
|
223
|
-
|
|
224
|
-
|
|
225
|
-
const prompt = await builder.build(resolved.value, {
|
|
226
|
-
resolvedBlobs,
|
|
227
|
-
tokenBudget: { total: 4000 }
|
|
228
|
-
});
|
|
229
|
-
|
|
230
|
-
console.log(prompt.system); // System prompt with persona, preferences, context
|
|
231
|
-
console.log(prompt.turns); // Conversation turns with resolved blobs
|
|
174
|
+
const data = new TextEncoder().encode('Content of a document');
|
|
175
|
+
const registerResult = await blobStore.register(data, 'text/plain', 'readme.txt');
|
|
176
|
+
if (registerResult.ok) {
|
|
177
|
+
const blobRef = registerResult.value;
|
|
178
|
+
// Use blobRef in a turn or context block
|
|
179
|
+
}
|
|
180
|
+
|
|
181
|
+
// Later, resolve the blob for a prompt
|
|
182
|
+
const resolveResult = await blobStore.resolveRef(blobRef, null);
|
|
183
|
+
if (resolveResult.ok) {
|
|
184
|
+
const resolved = resolveResult.value;
|
|
185
|
+
if (resolved.kind === 'inline') {
|
|
186
|
+
console.log(new TextDecoder().decode(resolved.data));
|
|
187
|
+
}
|
|
232
188
|
}
|
|
233
189
|
```
|
|
234
190
|
|
|
235
|
-
###
|
|
191
|
+
### Building a Prompt
|
|
236
192
|
|
|
237
|
-
|
|
193
|
+
`PromptBuilder` assembles a token‑aware prompt from an `EffectiveSession`.
|
|
238
194
|
|
|
239
195
|
```typescript
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
contextEntries: 20, // Max context entries in LRU cache
|
|
245
|
-
transcriptWindows: 3 // Max transcript windows in LRU cache
|
|
246
|
-
},
|
|
247
|
-
flush: {
|
|
248
|
-
maxBufferSize: 10, // Auto-flush after 10 turns
|
|
249
|
-
flushIntervalMs: 30000 // Auto-flush every 30 seconds
|
|
250
|
-
},
|
|
251
|
-
transcriptWindowSize: 20 // Default turns per window
|
|
196
|
+
import { PromptBuilder } from '@asaidimu/utils-workspace';
|
|
197
|
+
|
|
198
|
+
const builder = new PromptBuilder({
|
|
199
|
+
blobResolver: blobStore.resolveRefs.bind(blobStore),
|
|
252
200
|
});
|
|
253
|
-
```
|
|
254
201
|
|
|
255
|
-
|
|
202
|
+
const effective = await manager.resolveSession(workspace, 's1');
|
|
203
|
+
if (!effective.ok) throw new Error('Failed to resolve session');
|
|
256
204
|
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
|
|
205
|
+
const prompt = await builder.build(effective.value, {
|
|
206
|
+
tokenBudget: { total: 4000 },
|
|
207
|
+
relevanceConfig: { recentMessageWindow: 5, minScore: 0.2 },
|
|
208
|
+
providerId: 'anthropic', // for remote blobs
|
|
260
209
|
});
|
|
261
|
-
```
|
|
262
|
-
|
|
263
|
-
**IndexedDB Configuration**
|
|
264
210
|
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
211
|
+
console.log(prompt.system.instructions);
|
|
212
|
+
console.log(prompt.transcript.turns);
|
|
213
|
+
console.log(prompt.warnings);
|
|
268
214
|
```
|
|
269
215
|
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
## Project Architecture
|
|
273
|
-
|
|
274
|
-
### Core Components
|
|
275
|
-
|
|
276
|
-
**IndexState** — The in-memory representation of the workspace. Contains summaries of all entities (roles, preferences, contexts, sessions, blobs) and is designed for fast reads. State mutations produce patches that can be merged optimistically.
|
|
216
|
+
### Common Use Cases
|
|
277
217
|
|
|
278
|
-
|
|
218
|
+
#### 1. Offline‑First Chat Application
|
|
279
219
|
|
|
280
|
-
|
|
220
|
+
- Use `IndexedDBStorage` and `IndexedDBBlobStorage` for persistence.
|
|
221
|
+
- Set `eagerEviction: false` to keep blobs until manually cleaned.
|
|
222
|
+
- Let `ContentStore` auto‑flush when `maxBufferSize` is reached.
|
|
223
|
+
- Call `blobStore.gc()` occasionally to reclaim space.
|
|
281
224
|
|
|
282
|
-
|
|
225
|
+
#### 2. Multi‑Session Workspace with Topic‑Based Context
|
|
283
226
|
|
|
284
|
-
|
|
227
|
+
- Create context entries with topics.
|
|
228
|
+
- Use `session:topics:add` command to associate topics with a session.
|
|
229
|
+
- `resolveSession` automatically pulls in context entries that match those topics.
|
|
285
230
|
|
|
286
|
-
|
|
287
|
-
- **MemoryStorage** — In-memory implementation for testing
|
|
288
|
-
- **IndexedDBStorage** — Persistent browser storage for content
|
|
289
|
-
- **MemoryBlobStorage** — In-memory blob storage
|
|
290
|
-
- **IndexedDBBlobStorage** — Persistent browser storage for blobs
|
|
231
|
+
#### 3. Branching Conversations
|
|
291
232
|
|
|
292
|
-
|
|
233
|
+
- Use `session.branchFrom()` to fork a conversation tree.
|
|
234
|
+
- Each branch maintains its own head pointer.
|
|
235
|
+
- Fork an entire session with the `session:fork` command.
|
|
293
236
|
|
|
294
|
-
|
|
295
|
-
Command → WorkspaceManager.dispatch()
|
|
296
|
-
↓
|
|
297
|
-
WorkspaceReducer (pure patch generation)
|
|
298
|
-
↓
|
|
299
|
-
ContentStore (side effects: save roles, preferences, turns, etc.)
|
|
300
|
-
↓
|
|
301
|
-
Storage Backend (IndexedDB / Memory)
|
|
302
|
-
```
|
|
237
|
+
#### 4. Integrating with AI Providers
|
|
303
238
|
|
|
304
|
-
|
|
239
|
+
- After uploading a blob to a provider (e.g., Anthropic), record the remote ID:
|
|
240
|
+
```typescript
|
|
241
|
+
await blobStore.recordRemoteId(sha256, 'anthropic', 'file_abc123');
|
|
242
|
+
```
|
|
243
|
+
- In `PromptBuilder.build`, pass the `providerId` to get remote references instead of inline bytes.
|
|
305
244
|
|
|
306
|
-
|
|
307
|
-
IndexState + SessionId → ContentStore.resolveSession()
|
|
308
|
-
↓ (async content loading)
|
|
309
|
-
EffectiveSession → BlobStore.resolveRefs()
|
|
310
|
-
↓ (blob resolution)
|
|
311
|
-
PromptBuilder.build()
|
|
312
|
-
↓ (token budgeting, summarization)
|
|
313
|
-
Prompt (system, turns, warnings)
|
|
314
|
-
```
|
|
245
|
+
---
|
|
315
246
|
|
|
316
|
-
|
|
247
|
+
## Project Architecture
|
|
317
248
|
|
|
318
|
-
|
|
249
|
+
### Core Components
|
|
319
250
|
|
|
320
|
-
|
|
251
|
+
- **`WorkspaceManager`** – Facade that coordinates the pure reducer, content side effects, and session resolution.
|
|
252
|
+
- **`ContentStore`** – Manages roles, preferences, context, and transcript windows. Handles LRU caching, dirty buffers, and session activation.
|
|
253
|
+
- **`BlobStore`** – Owns the blob registry (ref counts, remote IDs) and SHA‑256 computation. Interacts with a `BlobStorage` backend.
|
|
254
|
+
- **`TurnTree`** – Pure logic for turn chains, subtree deletion, and head management. Used by `ContentStore` and `Session`.
|
|
255
|
+
- **`Session`** – Live object for an open session. Holds the turn DAG and dirty buffer, provides mutation methods (`addTurn`, `editTurn`, `branchFrom`, `deleteTurn`, `switchLeft`/`Right`).
|
|
256
|
+
- **`SessionManager`** – Entry point for opening and closing sessions; loads the turn DAG from storage.
|
|
257
|
+
- **`PromptBuilder`** – Orchestrates context retrieval, token planning, blob resolution, and final assembly.
|
|
258
|
+
- **Storage backends** – `MemoryStorage`, `IndexedDBStorage` (content), `MemoryBlobStorage`, `IndexedDBBlobStorage` (blobs).
|
|
321
259
|
|
|
322
|
-
|
|
323
|
-
interface Summarizer {
|
|
324
|
-
summarize(transcript: Turn[], tokenBudget: number): Promise<{
|
|
325
|
-
summary: string;
|
|
326
|
-
remaining: Turn[];
|
|
327
|
-
}>;
|
|
328
|
-
}
|
|
260
|
+
### Data Flow
|
|
329
261
|
|
|
330
|
-
|
|
331
|
-
|
|
262
|
+
1. **User action** → `WorkspaceManager.dispatch(command)`.
|
|
263
|
+
2. **Reducer** produces a `DeepPartial<Workspace>` patch and schedules side effects.
|
|
264
|
+
3. **Side effects** write to `ContentStore` (roles, preferences, context, turns via `TurnTree`).
|
|
265
|
+
4. **Session activation** loads the turn DAG via `TurnTree.buildNodeGraph()`.
|
|
266
|
+
5. **Turn mutations** update the in‑memory `Session.nodes` and dirty buffer; auto‑flush writes to storage when buffer size exceeds threshold.
|
|
267
|
+
6. **Prompt building** retrieves an `EffectiveSession` (via `Session.resolve` or `ContentStore.resolveSession`), ranks context, plans token usage, resolves blobs, and assembles the final `Prompt`.
|
|
332
268
|
|
|
333
|
-
|
|
269
|
+
### Extension Points
|
|
334
270
|
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
estimator: (text) => myProviderTokenCounter.count(text)
|
|
340
|
-
}
|
|
341
|
-
});
|
|
342
|
-
```
|
|
271
|
+
- **`ContextRetriever`** – Replace the default Jaccard‑based retriever with a vector or hybrid approach.
|
|
272
|
+
- **`TokenPlanner`** – Implement custom token estimation (e.g., using tiktoken) and priority strategies.
|
|
273
|
+
- **`Summarizer`** – Hook into an LLM to compress transcripts.
|
|
274
|
+
- **`BlobStorage` / `ContentStorage`** – Add new backends (e.g., S3, SQLite) by implementing the interface.
|
|
343
275
|
|
|
344
276
|
---
|
|
345
277
|
|
|
@@ -348,59 +280,51 @@ const prompt = await builder.build(session, {
|
|
|
348
280
|
### Development Setup
|
|
349
281
|
|
|
350
282
|
1. Clone the repository:
|
|
351
|
-
```bash
|
|
352
|
-
git clone https://github.com/asaidimu/utils
|
|
353
|
-
cd utils
|
|
354
|
-
```
|
|
355
|
-
|
|
283
|
+
```bash
|
|
284
|
+
git clone https://github.com/asaidimu/erp-utils.git
|
|
285
|
+
cd erp-utils/src/workspace
|
|
286
|
+
```
|
|
356
287
|
2. Install dependencies:
|
|
357
|
-
```bash
|
|
358
|
-
npm install
|
|
359
|
-
```
|
|
360
|
-
|
|
288
|
+
```bash
|
|
289
|
+
npm install
|
|
290
|
+
```
|
|
361
291
|
3. Build the project:
|
|
362
|
-
```bash
|
|
363
|
-
npm run build
|
|
364
|
-
```
|
|
292
|
+
```bash
|
|
293
|
+
npm run build
|
|
294
|
+
```
|
|
365
295
|
|
|
366
|
-
###
|
|
296
|
+
### Scripts
|
|
367
297
|
|
|
368
|
-
| Script
|
|
369
|
-
|
|
370
|
-
| `npm
|
|
371
|
-
| `npm run test` | Run
|
|
372
|
-
| `npm run test:
|
|
373
|
-
| `npm run lint` | Run ESLint |
|
|
374
|
-
| `npm run format` | Format code with Prettier |
|
|
375
|
-
| `npm run typecheck` | Run TypeScript type checking |
|
|
298
|
+
| Script | Description |
|
|
299
|
+
|----------------------|-------------------------------------------|
|
|
300
|
+
| `npm test` | Run tests once (Vitest) |
|
|
301
|
+
| `npm run test:watch` | Run tests in watch mode |
|
|
302
|
+
| `npm run test:browser` | Run tests in a browser environment |
|
|
376
303
|
|
|
377
304
|
### Testing
|
|
378
305
|
|
|
379
|
-
The test suite uses
|
|
306
|
+
The test suite uses `vitest` and `fake-indexeddb/auto` to simulate IndexedDB. Run the tests with:
|
|
380
307
|
|
|
381
308
|
```bash
|
|
382
|
-
npm
|
|
309
|
+
npm test
|
|
383
310
|
```
|
|
384
311
|
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
```bash
|
|
388
|
-
npm run test:coverage
|
|
389
|
-
```
|
|
312
|
+
All core functionality is covered. When contributing, please add tests for new features or bug fixes.
|
|
390
313
|
|
|
391
314
|
### Contributing Guidelines
|
|
392
315
|
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
5. **Submit PR** — Open a pull request with a clear description of changes
|
|
316
|
+
- **Branching**: Use feature branches off `main`.
|
|
317
|
+
- **Commit messages**: Follow conventional commits (e.g., `feat: add new retriever`).
|
|
318
|
+
- **Pull requests**: Ensure tests pass and add new tests for changes.
|
|
319
|
+
- **Code style**: The project uses Prettier and ESLint. Run formatting before committing.
|
|
398
320
|
|
|
399
321
|
### Issue Reporting
|
|
400
322
|
|
|
401
|
-
|
|
402
|
-
-
|
|
403
|
-
-
|
|
323
|
+
Please open an issue on [GitHub](https://github.com/asaidimu/erp-utils/issues) with:
|
|
324
|
+
- A clear description of the problem.
|
|
325
|
+
- Steps to reproduce.
|
|
326
|
+
- Expected and actual behaviour.
|
|
327
|
+
- Version of the library and environment (Node.js / browser).
|
|
404
328
|
|
|
405
329
|
---
|
|
406
330
|
|
|
@@ -408,36 +332,30 @@ npm run test:coverage
|
|
|
408
332
|
|
|
409
333
|
### Troubleshooting
|
|
410
334
|
|
|
411
|
-
|
|
335
|
+
| Issue | Solution |
|
|
336
|
+
|--------------------------------------------|--------------------------------------------------------------------------------------------|
|
|
337
|
+
| `Database not open` error | Call `open()` on the storage backend before using it. |
|
|
338
|
+
| Blob not found in prompt | Ensure the blob is still referenced (refCount > 0) and not evicted. Run `gc()` only when safe. |
|
|
339
|
+
| Turns missing after reload | Check that `flush()` was called before deactivation, or that `maxBufferSize` was reached. |
|
|
340
|
+
| Prompt builder returns warnings for blobs | Verify that the blob is registered and, if remote, that `recordRemoteId` was called. |
|
|
412
341
|
|
|
413
|
-
|
|
414
|
-
- `eagerEviction: false` in BlobStore config
|
|
415
|
-
- `gc()` is not called before resolution
|
|
416
|
-
- Register the blob again if it was evicted
|
|
342
|
+
### FAQ
|
|
417
343
|
|
|
418
|
-
**Q:
|
|
344
|
+
**Q: How does blob deduplication work?**
|
|
345
|
+
A: `BlobStore` computes SHA‑256 of the data. If a blob with the same hash already exists, only the ref count is incremented – bytes are never duplicated.
|
|
419
346
|
|
|
420
|
-
|
|
347
|
+
**Q: Can I use this library in a Node.js environment without IndexedDB?**
|
|
348
|
+
A: Yes. Use `MemoryStorage` and `MemoryBlobStorage` for in‑memory persistence. For long‑term storage, you would need to implement a custom backend (e.g., file system or SQLite).
|
|
421
349
|
|
|
422
|
-
**Q:
|
|
350
|
+
**Q: How do I manage token budgets for large transcripts?**
|
|
351
|
+
A: Use the `summarizer` option in `PromptBuilder` to compress older turns. The default `TokenPlanner` trims from the oldest turns when the budget is exceeded.
|
|
423
352
|
|
|
424
|
-
|
|
425
|
-
|
|
426
|
-
**Q: TypeScript errors with custom storage backends**
|
|
427
|
-
|
|
428
|
-
Import the interface types and implement all required methods:
|
|
429
|
-
|
|
430
|
-
```typescript
|
|
431
|
-
import type { ContentStorage, BlobStorage } from '@asaidimu/utils-workspace';
|
|
432
|
-
```
|
|
353
|
+
**Q: What happens if I delete a role that is used by a session?**
|
|
354
|
+
A: The reducer will reject the command with `INVALID_COMMAND`. You must first switch sessions to a different role or delete the sessions.
|
|
433
355
|
|
|
434
356
|
### License
|
|
357
|
+
This project is licensed under the [MIT License](https://github.com/asaidimu/erp-utils/blob/main/LICENSE.md).
|
|
435
358
|
|
|
436
|
-
|
|
437
|
-
|
|
438
|
-
See the [LICENSE](LICENSE) file for details.
|
|
439
|
-
|
|
440
|
-
---
|
|
359
|
+
### Acknowledgments
|
|
441
360
|
|
|
442
|
-
|
|
443
|
-
- [@asaidimu/utils-store](https://github.com/asaidimu/utils-store) — Deep merge utilities and store patterns
|
|
361
|
+
Built with inspiration from content‑addressed storage systems (IPFS, git) and conversation tree models. Special thanks to the open‑source community for tools like TypeScript, Vitest, and fake-indexeddb.
|