@asaidimu/utils-workspace 1.0.0

package/LICENSE.md

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Saidimu
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,443 @@
+# @asaidimu/utils-workspace
+
+Content-addressed workspace and conversation management for AI applications. Handles roles, preferences, context, transcripts, and binary blobs with a unified storage model.
+
+[![npm version](https://img.shields.io/npm/v/@asaidimu/utils-workspace.svg)](https://www.npmjs.com/package/@asaidimu/utils-workspace)
+[![license](https://img.shields.io/npm/l/@asaidimu/utils-workspace.svg)](LICENSE)
+[![build status](https://img.shields.io/github/actions/workflow/status/asaidimu/utils-workspace/ci.yml)](https://github.com/asaidimu/utils-workspace/actions)
+[![typescript](https://img.shields.io/badge/TypeScript-5.0+-blue.svg)](https://www.typescriptlang.org/)
+
+## Table of Contents
+
+- [Overview & Features](#overview--features)
+- [Installation & Setup](#installation--setup)
+- [Usage Documentation](#usage-documentation)
+- [Project Architecture](#project-architecture)
+- [Development & Contributing](#development--contributing)
+- [Additional Information](#additional-information)
+
+---
+
+## Overview & Features
+
+**@asaidimu/utils-workspace** provides a complete solution for managing conversational state in AI applications. Built on a content-addressed storage model, it handles everything from role definitions and user preferences to session transcripts and binary blobs. The library separates concerns between index state (fast, always in memory) and content storage (asynchronous, backend-agnostic), enabling efficient operation in both browser and Node.js environments.
+
+The system implements a command-reducer pattern where state mutations are expressed as commands that produce patches. This design enables optimistic UI updates, offline-first operation, and easy persistence snapshots. Blob management uses SHA-256 content addressing for automatic deduplication, ref counting, and remote ID tracking across providers like Anthropic or OpenAI.
+
+### Key Features
+
+- **Content-Addressed Blob Storage** — SHA-256 based deduplication, reference counting, and atomic storage with IndexedDB or memory backends
+- **Turn-Based Transcripts** — Versioned conversation trees with branching, editing, and subtree deletion
+- **Rich Content Blocks** — Support for text, images, documents, tool use, and thinking blocks with token estimation
+- **Preference & Context Management** — Conflict resolution with timestamps, topic-based filtering, and relevance scoring with freshness decay
+- **Prompt Building** — Token budget allocation, transcript summarization, and context ranking that produces provider-agnostic prompts
+- **Multiple Storage Backends** — IndexedDB for browsers, MemoryStorage for testing and server-side use
+- **TypeScript First** — Full type definitions with strict mode compatibility
+- **Offline-Ready** — Dirty buffer management with configurable flush strategies and optimistic updates
+
+---
+
+## Installation & Setup
+
+### Prerequisites
+
+- Node.js 18+ (for server-side usage) or modern browser with Web Crypto API support
+- TypeScript 5.0+ (optional, for type safety)
+
+### Installation
+
+```bash
+npm install @asaidimu/utils-workspace
+```
+
+or with yarn:
+
+```bash
+yarn add @asaidimu/utils-workspace
+```
+
+### Basic Setup
+
+```typescript
+import {
+  IndexedDBStorage,
+  ContentStore,
+  WorkspaceManager,
+  PromptBuilder,
+  MemoryBlobStorage,
+  BlobStore
+    merge,
+} from '@asaidimu/utils-workspace';
+
+// Initialize content storage (IndexedDB for persistence)
+const contentBackend = new IndexedDBStorage({ dbName: 'my-workspace' });
+await contentBackend.open();
+
+// Initialize blob storage (memory for this example, IndexedDB also available)
+const blobBackend = new MemoryBlobStorage();
+const blobStore = new BlobStore(blobBackend);
+await blobStore.init();
+
+// Create content store with default configuration
+const contentStore = new ContentStore(contentBackend);
+
+// Create workspace manager
+const manager = new WorkspaceManager(contentStore);
+
+// Initialize empty index state
+const emptyState = {
+  format: 'aiworkspace/4.0',
+  workspace: {
+    id: 'ws-1',
+    settings: { language: 'en', defaultRole: 'assistant' },
+    project: { name: 'My Project', owner: 'user' },
+    indexes: { sessions: {}, roles: {}, preferences: {}, context: {}, topics: {} }
+  },
+  roles: {},
+  blobs: {},
+  preferences: {},
+  context: {},
+  sessions: {}
+};
+```
+
+---
+
+## Usage Documentation
+
+### Basic Usage
+
+Create a role, session, and add a turn:
+
+```typescript
+let indexState = emptyState;
+
+// 1. Add a role
+const addRoleCmd = {
+  type: 'role:add' as const,
+  timestamp: new Date().toISOString(),
+  payload: {
+    id: 'role-1',
+    name: 'assistant',
+    label: 'Assistant',
+    persona: 'You are a helpful assistant.',
+    preferences: []
+  }
+};
+
+const roleResult = await manager.dispatch(indexState, addRoleCmd);
+if (roleResult.ok) {
+  // Merge the patch into your state
+  indexState = merge(indexState, roleResult.value) ;
+}
+
+// 2. Create a session
+const createSessionCmd = {
+  type: 'session:create' as const,
+  timestamp: new Date().toISOString(),
+  payload: {
+    id: 'session-1',
+    label: 'My First Conversation',
+    role: 'assistant',
+    topics: ['general'],
+    preferences: []
+  }
+};
+
+const sessionResult = await manager.dispatch(indexState, createSessionCmd);
+if (sessionResult.ok) {
+  indexState = merge(indexState, sessionResult.value) ;
+}
+
+// 3. Activate the session (enables dirty buffer for performance)
+await manager.activateSession(indexState, 'session-1');
+
+// 4. Add a turn
+const addTurnCmd = {
+  type: 'turn:add' as const,
+  timestamp: new Date().toISOString(),
+  payload: {
+    sessionId: 'session-1',
+    turn: {
+      id: 'turn-1',
+      version: 0,
+      role: 'user',
+      blocks: [{ type: 'text', text: 'Hello, how are you?' }],
+      timestamp: new Date().toISOString(),
+      roleSnapshot: 'assistant',
+      parent: null
+    }
+  }
+};
+
+await manager.dispatch(indexState, addTurnCmd);
+await manager.flush(); // Persist dirty buffer to storage
+```
+
+### Registering and Using Blobs
+
+```typescript
+// Register a blob (e.g., an image or document)
+const imageData = new TextEncoder().encode('fake-image-data');
+const blobResult = await blobStore.register(imageData, 'image/jpeg', 'photo.jpg');
+
+if (blobResult.ok) {
+  const blobRef = blobResult.value;
+  
+  // Use the blob in a turn
+  const turnWithBlob = {
+    id: 'turn-2',
+    version: 0,
+    role: 'user',
+    blocks: [
+      { type: 'text', text: 'What is in this image?' },
+      { type: 'image', blob: blobRef }
+    ],
+    timestamp: new Date().toISOString(),
+    roleSnapshot: 'assistant',
+    parent: null
+  };
+  
+  await manager.dispatch(indexState, {
+    type: 'turn:add',
+    timestamp: new Date().toISOString(),
+    payload: { sessionId: 'session-1', turn: turnWithBlob }
+  });
+}
+```
+
+### Building Prompts
+
+```typescript
+// Resolve the session to get effective preferences, context, and transcript
+const resolved = await manager.resolveSession(indexState, 'session-1');
+
+if (resolved.ok) {
+  // Resolve all blobs in the session
+  const refs = resolved.value.transcript
+    .flatMap(t => t.blocks.filter(b => b.type === 'image' || b.type === 'document'))
+    .map(b => b.blob);
+  
+  const { resolved: resolvedBlobs, errors } = await blobStore.resolveRefs(refs, null);
+  
+  // Build the prompt
+  const builder = new PromptBuilder();
+  const prompt = await builder.build(resolved.value, {
+    resolvedBlobs,
+    tokenBudget: { total: 4000 }
+  });
+  
+  console.log(prompt.system); // System prompt with persona, preferences, context
+  console.log(prompt.turns);   // Conversation turns with resolved blobs
+}
+```
+
+### Configuration
+
+**ContentStore Configuration**
+
+```typescript
+const contentStore = new ContentStore(backend, {
+  cache: {
+    roles: 10,              // Max roles in LRU cache
+    preferences: 50,        // Max preferences in LRU cache
+    contextEntries: 20,     // Max context entries in LRU cache
+    transcriptWindows: 3    // Max transcript windows in LRU cache
+  },
+  flush: {
+    maxBufferSize: 10,      // Auto-flush after 10 turns
+    flushIntervalMs: 30000  // Auto-flush every 30 seconds
+  },
+  transcriptWindowSize: 20  // Default turns per window
+});
+```
+
+**BlobStore Configuration**
+
+```typescript
+const blobStore = new BlobStore(backend, {
+  eagerEviction: false  // When false, blobs with refCount 0 persist until gc()
+});
+```
+
+**IndexedDB Configuration**
+
+```typescript
+const contentBackend = new IndexedDBStorage({ dbName: 'custom-db-name' });
+const blobBackend = new IndexedDBBlobStorage({ dbName: 'custom-blob-db' });
+```
+
+---
+
+## Project Architecture
+
+### Core Components
+
+**IndexState** — The in-memory representation of the workspace. Contains summaries of all entities (roles, preferences, contexts, sessions, blobs) and is designed for fast reads. State mutations produce patches that can be merged optimistically.
+
+**ContentStore** — Manages asynchronous access to persisted content. Implements LRU caching with pinning for active sessions, dirty buffer management for turns, and maintains the distinction between index state and full content.
+
+**WorkspaceManager** — The public facade that coordinates between commands, reducer, and content persistence. Handles side effects (saving to storage) while keeping the reducer pure.
+
+**BlobStore** — Content-addressed blob management with SHA-256 deduplication, reference counting, remote ID tracking, and resolution between inline and remote blob references.
+
+**PromptBuilder** — Transforms an EffectiveSession into a provider-agnostic prompt structure with token budgeting, preference conflict resolution, context ranking, and optional summarization.
+
+**Storage Backends** — Pluggable storage layers:
+- **MemoryStorage** — In-memory implementation for testing
+- **IndexedDBStorage** — Persistent browser storage for content
+- **MemoryBlobStorage** — In-memory blob storage
+- **IndexedDBBlobStorage** — Persistent browser storage for blobs
+
+### Data Flow
+
+```
+Command → WorkspaceManager.dispatch()
+    ↓
+WorkspaceReducer (pure patch generation)
+    ↓
+ContentStore (side effects: save roles, preferences, turns, etc.)
+    ↓
+Storage Backend (IndexedDB / Memory)
+```
+
+For resolution and prompt building:
+
+```
+IndexState + SessionId → ContentStore.resolveSession()
+    ↓ (async content loading)
+EffectiveSession → BlobStore.resolveRefs()
+    ↓ (blob resolution)
+PromptBuilder.build()
+    ↓ (token budgeting, summarization)
+Prompt (system, turns, warnings)
+```
+
+### Extension Points
+
+**Custom Storage Backends** — Implement the `ContentStorage` or `BlobStorage` interfaces to support alternative persistence (SQLite, cloud storage, etc.)
+
+**Summarizer** — Inject a custom summarizer into `PromptBuilder` for transcript compression:
+
+```typescript
+interface Summarizer {
+  summarize(transcript: Turn[], tokenBudget: number): Promise<{
+    summary: string;
+    remaining: Turn[];
+  }>;
+}
+
+const builder = new PromptBuilder(myCustomSummarizer);
+```
+
+**Token Estimators** — Override the default text estimator for provider-specific token counting:
+
+```typescript
+const prompt = await builder.build(session, {
+  tokenBudget: {
+    total: 8000,
+    estimator: (text) => myProviderTokenCounter.count(text)
+  }
+});
+```
+
+---
+
+## Development & Contributing
+
+### Development Setup
+
+1. Clone the repository:
+```bash
+git clone https://github.com/asaidimu/utils-workspace.git
+cd utils-workspace
+```
+
+2. Install dependencies:
+```bash
+npm install
+```
+
+3. Build the project:
+```bash
+npm run build
+```
+
+### Available Scripts
+
+| Script | Description |
+|--------|-------------|
+| `npm run build` | Compile TypeScript to JavaScript |
+| `npm run test` | Run all tests with Vitest |
+| `npm run test:watch` | Run tests in watch mode |
+| `npm run lint` | Run ESLint |
+| `npm run format` | Format code with Prettier |
+| `npm run typecheck` | Run TypeScript type checking |
+
+### Testing
+
+The test suite uses Vitest with fake-indexeddb for browser environment simulation. Run tests with:
+
+```bash
+npm run test
+```
+
+Coverage reports can be generated with:
+
+```bash
+npm run test:coverage
+```
+
+### Contributing Guidelines
+
+1. **Fork and Branch** — Create a feature branch from `main`
+2. **Write Tests** — Add tests for new functionality or bug fixes
+3. **Update Documentation** — Keep README and JSDoc comments current
+4. **Commit Convention** — Use conventional commits (feat:, fix:, docs:, etc.)
+5. **Submit PR** — Open a pull request with a clear description of changes
+
+### Issue Reporting
+
+- **Bug Reports** — Include minimal reproduction, expected vs actual behavior, and environment details
+- **Feature Requests** — Describe use case and expected API
+- **Security Issues** — Email maintainers directly (see package.json for contact)
+
+---
+
+## Additional Information
+
+### Troubleshooting
+
+**Q: Blob bytes not found locally error**
+
+Blobs with refCount 0 are eligible for garbage collection. If you need them later, ensure:
+- `eagerEviction: false` in BlobStore config
+- `gc()` is not called before resolution
+- Register the blob again if it was evicted
+
+**Q: IndexedDB open fails with "blocked" error**
+
+Another browser tab has an open connection to the same database. Close the other tab or wait for it to release the connection.
+
+**Q: Turn edits not persisting**
+
+Ensure `flush()` is called after batch operations, or configure `flush.maxBufferSize` to auto-flush.
+
+**Q: TypeScript errors with custom storage backends**
+
+Import the interface types and implement all required methods:
+
+```typescript
+import type { ContentStorage, BlobStorage } from '@asaidimu/utils-workspace';
+```
+
+### License
+
+MIT © [Saidimu](https://github.com/asaidimu)
+
+See the [LICENSE](LICENSE) file for details.
+
+---
+
+**Related Projects**
+- [@asaidimu/utils-store](https://github.com/asaidimu/utils-store) — Deep merge utilities and store patterns